{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Writing Low-Level TensorFlow Code\n",
    "\n",
    "\n",
    "**Learning Objectives**\n",
    "\n",
    " 1. Practice defining and performing basic operations on constant Tensors\n",
    " 2. Use Tensorflow's automatic differentiation capability\n",
    " 3. Learn how to train a linear regression from scratch with TensorFLow\n",
    "\n",
    "\n",
    "## Introduction \n",
    "\n",
    "In this notebook, we will start by reviewing the main operations on Tensors in TensorFlow and understand how to manipulate TensorFlow Variables. We explain how these are compatible with python built-in list and numpy arrays. \n",
    "\n",
    "Then we will jump to the problem of training a linear regression from scratch with gradient descent. The first order of business will be to understand how to compute the gradients of a function (the loss here) with respect to some of its arguments (the model weights here). The TensorFlow construct allowing us to do that is `tf.GradientTape`, which we will describe. \n",
    "\n",
    "At last we will create a simple training loop to learn the weights of a 1-dim linear regression using synthetic data generated from a linear model. \n",
    "\n",
    "As a bonus exercise, we will do the same for data generated from a non linear model, forcing us to manual engineer non-linear features to improve our linear model performance.\n",
    "\n",
    "Each learning objective will correspond to a __#TODO__  in this student lab notebook -- try to complete this notebook first and then review the [solution notebook](https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/introduction_to_tensorflow/solutions/write_low_level_code.ipynb)\n"  ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "2.5.0\n"
      ]
     }
   ],
   "source": [
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Operations on Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variables and Constants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensors in TensorFlow are either contant (`tf.constant`) or variables (`tf.Variable`).\n",
    "Constant values can not be changed, while variables values can be.\n",
    "\n",
    "The main difference is that instances of `tf.Variable` have methods allowing us to change \n",
    "their values while tensors constructed with `tf.constant` don't have these methods, and\n",
    "therefore their values can not be changed. When you want to change the value of a `tf.Variable`\n",
    "`x` use one of the following method: \n",
    "\n",
    "* `x.assign(new_value)`\n",
    "* `x.assign_add(value_to_be_added)`\n",
    "* `x.assign_sub(value_to_be_subtracted`\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(3,), dtype=int32, numpy=array([2, 3, 4], dtype=int32)>\n"
      ]
     }
   ],
   "source": [
    "x = tf.constant([2, 3, 4])\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.Variable(2.0, dtype=tf.float32, name='my_variable')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Variable 'my_variable:0' shape=() dtype=float32, numpy=45.8>\n"
      ]
     }
   ],
   "source": [
    "x.assign(45.8)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Variable 'my_variable:0' shape=() dtype=float32, numpy=49.8>\n"
      ]
     }
   ],
   "source": [
    "x.assign_add(4) \n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Variable 'my_variable:0' shape=() dtype=float32, numpy=46.8>\n"
      ]
     }
   ],
   "source": [
    "x.assign_sub(3)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Point-wise operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensorflow offers similar point-wise tensor operations as numpy does:\n",
    "    \n",
    "* `tf.add` allows to add the components of a tensor \n",
    "* `tf.multiply` allows us to multiply the components of a tensor\n",
    "* `tf.subtract` allow us to substract the components of a tensor\n",
    "* `tf.math.*` contains the usual math operations to be applied on the components of a tensor\n",
    "* and many more...\n",
    "\n",
    "Most of the standard arithmetic operations (`tf.add`, `tf.subtract`, etc.) are overloaded by the usual corresponding arithmetic symbols (`+`, `-`, etc.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Lab Task #1:** Performing basic operations on Tensors \n",
    "1. In the first cell, define two constants `a` and `b` and compute their sum in c and d respectively, below using `tf.add` and `+` and verify both operations produce the same values.\n",
    "2. In the second cell, compute the product of the constants `a` and `b` below using `tf.multiply` and `*` and verify both operations produce the same values.\n",
    "3. In the third cell, compute the exponential of the constant `a` using `tf.math.exp`. Note, you'll need to specify the type for this operation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "c: tf.Tensor([ 8  2 10], shape=(3,), dtype=int32)\n",
       "d: tf.Tensor([ 8  2 10], shape=(3,), dtype=int32)\n"
      ]
     }
   ],
   "source": [
    "# TODO 1a\n",
    "a = # TODO -- Your code here.\n",
    "b = # TODO -- Your code here.\n",
    "c = # TODO -- Your code here.\n",
    "d = # TODO -- Your code here.\n",
    "\n",
    "print(\"c:\", c)\n",
    "print(\"d:\", d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "c: tf.Tensor([15 -3 16], shape=(3,), dtype=int32)\n",
       "d: tf.Tensor([15 -3 16], shape=(3,), dtype=int32)\n"
      ]
     }
   ],
   "source": [
    "# TODO 1b\n",
    "a = # TODO -- Your code here.\n",
    "b = # TODO -- Your code here.\n",
    "c = # TODO -- Your code here.\n",
    "d = # TODO -- Your code here.\n",
    "\n",
    "print(\"c:\", c)\n",
    "print(\"d:\", d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "b: tf.Tensor([ 148.41316    20.085537 2980.958   ], shape=(3,), dtype=float32)\n"
      ]
     }
   ],
   "source": [
    "# TODO 1c\n",
    "# tf.math.exp expects floats so we need to explicitly give the type\n",
    "a = # TODO -- Your code here.\n",
    "b = # TODO -- Your code here.\n",
    "\n",
    "print(\"b:\", b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NumPy Interoperability\n",
    "\n",
    "In addition to native TF tensors, tensorflow operations can take native python types and NumPy arrays as operands. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# native python list\n",
    "a_py = [1, 2] \n",
    "b_py = [3, 4] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([4, 6], dtype=int32)>\n"
      ]
     }
   ],
   "source": [
    "tf.add(a_py, b_py)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# numpy arrays\n",
    "a_np = np.array([1, 2])\n",
    "b_np = np.array([3, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(2,), dtype=int64, numpy=array([4, 6])>\n"
      ]
     }
   ],
   "source": [
    "tf.add(a_np, b_np) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# native TF tensor\n",
    "a_tf = tf.constant([1, 2])\n",
    "b_tf = tf.constant([3, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([4, 6], dtype=int32)>\n"
      ]
     }
   ],
   "source": [
    "tf.add(a_tf, b_tf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can convert a native TF tensor to a NumPy array using .numpy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "array([1, 2], dtype=int32)\n"
      ]
     }
   ],
   "source": [
    "a_tf.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression\n",
    "\n",
    "Now let's use low level tensorflow operations to implement linear regression.\n",
    "\n",
    "Later in the course you'll see abstracted ways to do this using high level TensorFlow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Toy Dataset\n",
    "\n",
    "We'll model the following function:\n",
    "\n",
    "\\begin{equation}\n",
    "y= 2x + 10\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "X:[0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]\n",
       "Y:[10. 12. 14. 16. 18. 20. 22. 24. 26. 28.]\n"
      ]
     }
   ],
   "source": [
    "X = tf.constant(range(10), dtype=tf.float32)\n",
    "Y = 2 * X + 10\n",
    "\n",
    "print(\"X:{}\".format(X))\n",
    "print(\"Y:{}\".format(Y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also create a test dataset to evaluate our models:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "X_test:[10. 11. 12. 13. 14. 15. 16. 17. 18. 19.]\n",
       "Y_test:[30. 32. 34. 36. 38. 40. 42. 44. 46. 48.]\n"
      ]
     }
   ],
   "source": [
    "X_test = tf.constant(range(10, 20), dtype=tf.float32)\n",
    "Y_test = 2 * X_test + 10\n",
    "\n",
    "print(\"X_test:{}\".format(X_test))\n",
    "print(\"Y_test:{}\".format(Y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loss Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simplest model we can build is a model that for each value of x returns the sample mean of the training set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_mean = Y.numpy().mean()\n",
    "\n",
    "\n",
    "def predict_mean(X):\n",
    "    y_hat = [y_mean] * len(X)\n",
    "    return y_hat\n",
    "\n",
    "Y_hat = predict_mean(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using mean squared error, our loss is:\n",
    "\\begin{equation}\n",
    "MSE = \\frac{1}{m}\\sum_{i=1}^{m}(\\hat{Y}_i-Y_i)^2\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this simple model the loss is then:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "33.0\n"
      ]
     }
   ],
   "source": [
    "errors = (Y_hat - Y)**2\n",
    "loss = tf.reduce_mean(errors)\n",
    "loss.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This values for the MSE loss above will give us a baseline to compare how a more complex model is doing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, if $\\hat{Y}$ represents the vector containing our model's predictions when we use a linear regression model\n",
    "\\begin{equation}\n",
    "\\hat{Y} = w_0X + w_1\n",
    "\\end{equation}\n",
    "\n",
    "we can write a loss function taking as arguments the coefficients of the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_mse(X, Y, w0, w1):\n",
    "    Y_hat = w0 * X + w1\n",
    "    errors = (Y_hat - Y)**2\n",
    "    return tf.reduce_mean(errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Function\n",
    "\n",
    "To use gradient descent we need to take the partial derivatives of the loss function with respect to each of the weights. We could manually compute the derivatives, but with Tensorflow's automatic differentiation capabilities we don't have to!\n",
    "\n",
    "During gradient descent we think of the loss as a function of the parameters $w_0$ and $w_1$. Thus, we want to compute the partial derivative with respect to these variables. \n",
    "\n",
    "For that we need to wrap our loss computation within the context of `tf.GradientTape` instance which will record gradient information:\n",
    "\n",
    "```python\n",
    "with tf.GradientTape() as tape:\n",
    "    loss = # computation \n",
    "```\n",
    "\n",
    "This will allow us to later compute the gradients of any tensor computed within the `tf.GradientTape` context with respect to instances of `tf.Variable`:\n",
    "\n",
    "```python\n",
    "gradients = tape.gradient(loss, [w0, w1])\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We illustrate this procedure by computing the loss gradients with respect to the model weights:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Lab Task #2:** Complete the function below to compute the loss gradients with respect to the model weights `w0` and `w1`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 2\n",
    "def compute_gradients(X, Y, w0, w1):\n",
    "    # TODO -- Your code here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w0 = tf.Variable(0.0)\n",
    "w1 = tf.Variable(0.0)\n",
    "\n",
    "dw0, dw1 = compute_gradients(X, Y, w0, w1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "dw0: -204.0\n"
      ]
     }
   ],
   "source": [
    "print(\"dw0:\", dw0.numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "dw1 -38.0\n"
      ]
     }
   ],
   "source": [
    "print(\"dw1\", dw1.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training Loop\n",
    "\n",
    "Here we have a very simple training loop that converges. Note we are ignoring best practices like batching, creating a separate test set, and random weight initialization for the sake of simplicity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Lab Task #3:** Complete the `for` loop below to train a linear regression. \n",
    "1. Use `compute_gradients` to compute `dw0` and `dw1`.\n",
    "2. Then, re-assign the value of `w0` and `w1` using the `.assign_sub(...)` method with the computed gradient values and the `LEARNING_RATE`.\n",
    "3. Finally, for every 100th step , we'll compute and print the `loss`. Use the `loss_mse` function we created above to compute the `loss`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "STEP 0 - loss: 35.70719528198242, w0: 4.079999923706055, w1: 0.7599999904632568\n",
       "STEP 100 - loss: 2.6017532348632812, w0: 2.4780430793762207, w1: 7.002389907836914\n",
       "STEP 200 - loss: 0.26831889152526855, w0: 2.153517961502075, w1: 9.037351608276367\n",
       "STEP 300 - loss: 0.027671903371810913, w0: 2.0493006706237793, w1: 9.690855979919434\n",
       "STEP 400 - loss: 0.0028539239428937435, w0: 2.0158326625823975, w1: 9.90071964263916\n",
       "STEP 500 - loss: 0.0002943490108009428, w0: 2.005084753036499, w1: 9.96811580657959\n",
       "STEP 600 - loss: 3.0356444767676294e-05, w0: 2.0016329288482666, w1: 9.989760398864746\n",
       "STEP 700 - loss: 3.1322738323069643e-06, w0: 2.0005245208740234, w1: 9.996710777282715\n",
       "STEP 800 - loss: 3.2238213520940917e-07, w0: 2.0001683235168457, w1: 9.998944282531738\n",
       "STEP 900 - loss: 3.369950718479231e-08, w0: 2.000054359436035, w1: 9.999658584594727\n",
       "STEP 1000 - loss: 3.6101481803996194e-09, w0: 2.0000178813934326, w1: 9.99988842010498\n"
      ]
     }
   ],
   "source": [
    "# TODO 3\n",
    "STEPS = 1000\n",
    "LEARNING_RATE = .02\n",
    "MSG = \"STEP {step} - loss: {loss}, w0: {w0}, w1: {w1}\\n\"\n",
    "\n",
    "\n",
    "w0 = tf.Variable(0.0)\n",
    "w1 = tf.Variable(0.0)\n",
    "\n",
    "\n",
    "for step in range(0, STEPS + 1):\n",
    "\n",
    "    dw0, dw1 = # TODO -- Your code here.\n",
    "\n",
    "    if step % 100 == 0:\n",
    "        loss = # TODO -- Your code here.\n",
    "        print(MSG.format(step=step, loss=loss, w0=w0.numpy(), w1=w1.numpy()))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's compare the test loss for this linear regression to the test loss from the baseline model that outputs always the mean of the training set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "2.4563633e-08\n"
      ]
     }
   ],
   "source": [
    "loss = loss_mse(X_test, Y_test, w0, w1)\n",
    "loss.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is indeed much better!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try modeling a non-linear function such as: $y=xe^{-x^2}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = tf.constant(np.linspace(0, 2, 1000), dtype=tf.float32)\n",
    "Y = X * tf.exp(-X**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoOklEQVR4nO3deXhU5d3/8fd3spJANkjYAgQIEMIaCIiIC2gVEIsrAipulVKlLq3bo7bPr9qnra37iqCoqEDdUAQUd6EgS1gSCGsIhKwkEJIQsif3748ETSPLJMzkzPJ9XReXycwZ5uNw+HByn/vcR4wxKKWUcn82qwMopZRyDC10pZTyEFroSinlIbTQlVLKQ2ihK6WUh/C16o07dOhgYmJirHp7pZRyS5s2bTpsjIk82XOWFXpMTAxJSUlWvb1SSrklEck41XM65KKUUh5CC10ppTyEFrpSSnkILXSllPIQWuhKKeUhtNCVUspDaKErpZSHsGweuvI8NbV1ZBeVk374OAUllRSXV3O8qgZfm+DrYyM8yI+OIYF0CWtDTPtg/H31eEIpR9JCVy1WU1vHhv2FrE47zMb9haRkF1NVU2fXa/19bPTr1I7B0aGc3yeS82Lb0y7Qz8mJlfJsWuiq2VKyili0IZOVqXkUHq/C1yYM7BrKjFE96NuxHb0ig+kYEkhokB/B/r7U1hmqa+soPF7FoZIKsovK2ZFbQmp2CZ9uzeG99QfxtQnn9m7PlUO7Mn5gJ4IDdNdUqrnEqjsWJSYmGr30333U1RlWbM9l3ur9JGcWEeTvwyX9OzJxUCcu6BtJkH/LCri6to7NGUf5fk8By1NyOVhYRhs/H65M6MrtY2KIjWrn4P8TpdybiGwyxiSe9DktdHU6xhi+2nGIZ77aw668Y/SKDGbGqB5cPTyaEAcPkRhj2JRxlA+SsvhkazaVNXWM7RfJPZf0ZWi3MIe+l1LuSgtdtUh6QSl/+nQ7a9KO0LNDMPde0odJg7vgYxOnv/eR0kreW3+Qt9YeoPB4FZcN6Mj9l/ajT0c9YlfeTQtdNUt1bR0vf5fGK9/tI8DPxoOX9WPayO74+rT+rJTSyhreWL2feavTKauq4aZRPfjDpf0IbaMnUJV30kJXdss4cpy7F20hOauYyUO78Ojl/YlqF2h1LAqPV/Hc13t4d10GEcEBPHZ5fyYP7YKI839aUMqVaKEru3y6NZtHPt6Gj034xzWDmTios9WRfmFbVjGPfbKN5KxixsVF8Y9rBrnEPzhKtZbTFbpe2aGorTP8/fOd3LN4K/FdQvj83gtcsswBBkWH8vGd5/HnSfGsSTvMZc+u4vNtuVbHUsolaKF7uWMV1cxckMRrP6Rz46juLLxjFF3D2lgd67R8bMJtY3qy/O4xRIcH8bv3NvPAB8mUV9VaHU0pS9lV6CIyXkR2i0iaiDx8mu1GiEitiFzruIjKWY6UVjJt3jq+31PAE5MH8NcrB+FnwYnPloqNasfHd45m9thYPtycxdWvruXA4eNWx1LKMmf82ysiPsDLwAQgHpgmIvGn2O5JYKWjQyrHyykq57rXfiQtv5TXb07kpnNjrI7UIn4+Nu6/rB/zbxlBbnE5V7z0H75MzbM6llKWsOdwbCSQZoxJN8ZUAYuBySfZ7vfAR0C+A/MpJzh4pIzr5vxIQUklC247h7H9oqyOdNbG9ovis9ljiGkfzMx3NvHyd2lYdcJfKavYU+hdgcxG32c1PPYTEekKXAXMcVw05Qw5ReVMm7eO41U1LJo5ipE9I6yO5DDdIoL4YNa5TB7ahX+t3M2DH6bYvViYUp7AngU4TjbRt+mhz3PAQ8aY2tPNCxaRmcBMgO7du9sZUTlK/rEKbnh9PSXl1Sy8YxQDu4ZaHcnhAv18eO76ocS0D+b5b/aSebSMOTcOJyzI3+poSjmdPUfoWUC3Rt9HAzlNtkkEFovIAeBa4BURubLpb2SMmWuMSTTGJEZGRrYssWqRorIqbnp9A3nFFbx12wgGRXtemZ8gItz3q748d/1QNmcUce2cH8ktLrc6llJOZ0+hbwT6iEhPEfEHpgJLG29gjOlpjIkxxsQAHwJ3GmM+cXRY1TKVNbXMXLCJ/YeP8/rNiQzv4TnDLKdzZUJXFtw+krziCq599Uf26wwY5eHOWOjGmBpgNvWzV3YC7xtjUkVklojMcnZAdXaMMTz80TY2HCjkX9cN5rzYDlZHalWjerVn8cxRlFfXct2ctaTmFFsdSSmn0Uv/PdxzX+/hua/3cv+lfZk9ro/VcSyTll/KjDfWc6yyhjdvGUFijHf8lKI8j17676U+S87hua/3cu3waO4aG2t1HEvFRrXlg9+NJrJtADPmb2DjgUKrIynlcFroHmrPoWM8+GEKiT3C+dtVg3RVQqBrWBsWzxxFp5BAbpm/gU0ZWurKs2ihe6BjFdXMemcTbQN9eeWGYfj76h/zCVEhgSyaOYqokEBunr+RTRlHrY6klMPo33QPY4zh/g+SySgs4+Xpw4gK0aVlm+oYEsiiO0YR2S6Am+dvYPNBLXXlGbTQPcwb/9nPytRD/M+EOI+6CtTROoXWl3qHtv7cPH+Dzn5RHkEL3YOk5hTzzy9286v4jtw+pqfVcVxep9BAFt4xinYBvtw8f6Ou1Kjcnha6hyivquWexVsJC/LjyWsG60lQO3UJa8OC28+hzhhufGM9ecUVVkdSqsW00D3E31bsJC2/lKenDCEiWNctaY7YqLa8desIjh6vYsb89RSVVVkdSakW0UL3AN/sPMQ76zL4zZienN9H18hpicHRYcy7OZEDh8u45c2NHK+ssTqSUs2mhe7misqqeOijbcR1ascD4/tZHcetje7dgRenJ5CSVcRdCzdTU6tL7yr3ooXu5h5ftoOisiqenjKEAF8fq+O4vcsGdOKvVw7i+90F/O/SVL1JhnIr9qyHrlzUd7vz+XhzNrPHxjKgi+cuh9vapp/TnYOFZcz5YR/dIoKYdWFvqyMpZRctdDd1rKKaRz/eRmxUW35/sXev0+IMD17Wj6yjZfzj811Eh7dh0uAuVkdS6oy00N3Uk1/sIrekgg9njdahFiew2YSnrhvCoZIK/vB+Mp1CAnWFRuXydAzdDW3KKOTddQe5dXRPhvcItzqOxwr082HuTYl0DWvDHQuS9AYZyuVpobuZmto6Hvsklc6hgfzx0r5Wx/F44cH+vHXrCAB+8/ZGSiqqLU6k1KlpobuZd9ZlsDO3hD9Piic4QEfMWkOP9sG8euNwMo6U8fuFW6it05kvyjVpobuR/GMVPPPlHs7v04HxAztZHcerjOrVnscnD+SHPQX84/OdVsdR6qT0EM+N/H3FLipr6nh88kBdq8UC08/pzu68Euat3k+/TiFcOzza6khK/Rc9QncT69OPsGRLNjMv6EXPDsFWx/Faf5oUz3mx7Xnk4216xyPlcrTQ3UBdneHxZTvoGtbG6+8NajVfHxsvTx9Gl7BAfvvOJrKLyq2OpNRPtNDdwMdbsknNKeHB8f1o469zzq0WFuTP6zcnUlldx8wFSVRU11odSSlAC93llVfV8tTK3QyJDuUKvVrRZcRGteP5aUNJzSnhkSXbdM0X5RK00F3cvNXp5JVU8NikeGw2PRHqSsbFdeTeS/rw8eZs3l2XYXUcpbTQXVl+SQVzftjHhIGdGKGXnbuku8f1YVxcFI8v26EnSZXltNBd2NNf7qG6to6HJ8RZHUWdgs0mPDtlKF3C2vC7dzeTf0xvYaeso4XuonbnHeP9TZnMODeGHu11mqIrCw3yY86NwympqGb2e1uo1htjKItoobuop7/cTVt/X2brNEW30L9zCE9eM5gNBwr52wq9klRZQwvdBSVnFvHljkPccUEvwvWGz25j8tCu3HpeDG+uOcCnW7OtjqO8kBa6C3rqy91EBPtz25ieVkdRzfTIxP6MjIngoY9S2JVXYnUc5WW00F3Mj/uOsHrvYe68qDdtdTVFt+PnY+OlGxJoF+jHne9uprSyxupIyotoobsQYwxPfbmbjiEB3Diqh9VxVAtFtQvkxWkJHDhynIc/StGLjlSr0UJ3Id/tzmdTxlHuvrgPgX56ib87G9WrPX+8tB/LUnL1oiPVarTQXYQxhqe/3EP3iCCmJHazOo5ygN9d2Jux/SJ5YtlOUrKKrI6jvIAWuov4dlc+qTkl/H5cLH4++sfiCWw24ZkpQ+nQ1p8739tMcZnevk45lzaHCzDG8MK3aUSHt+HKhK5Wx1EOFB7sz0s3DONQSQV//CBZx9OVU2mhu4DVew+TnFnEnRfp0bknGtY9nP+Z0J+vdx5i3up0q+MoD2ZXe4jIeBHZLSJpIvLwSZ6fLCIpIrJVRJJEZIzjo3omYwwvfruXzqGBXDNcj8491a3nxTBhYCee/GI3SQd0ES/lHGcsdBHxAV4GJgDxwDQRiW+y2TfAEGPMUOA24HUH5/RY69IL2XjgKLMu7E2Ar85s8VQiwpPXDiY6vA2zF27hSGml1ZGUB7LnCH0kkGaMSTfGVAGLgcmNNzDGlJqfBweDAR0otNML3+wlsl0A14/QmS2eLiTQj1duGEZhWRX3/nsrtXX610Q5lj2F3hXIbPR9VsNj/0VErhKRXcBy6o/S1RkkHSjkx/Qj/PaCXjrv3EsM6BLKX349gNV7D/Pq92lWx1Eexp5CP9ltcn5xaGGMWWKMiQOuBJ446W8kMrNhjD2poKCgWUE90UvfpRER7M/0c7pbHUW1oqkjuvHrIV145qs9bNiv4+nKcewp9Cyg8XhANJBzqo2NMauA3iLS4STPzTXGJBpjEiMjI5sd1pPszC3h+90F3HZeDEH+umaLNxER/u+qgXSPCOLuRVsoPF5ldSTlIewp9I1AHxHpKSL+wFRgaeMNRCRWRKTh62GAP3DE0WE9ybxV6QT5++iaLV6qXaAfL00fRuHxKu7/IJk6HU9XDnDGQjfG1ACzgZXATuB9Y0yqiMwSkVkNm10DbBeRrdTPiLne6BUUp5RTVM7S5ByuH9GNsCBd79xbDewayqOX9+fbXfm88Z/9VsdRHsCun/WNMSuAFU0em9Po6yeBJx0bzXO9uWY/Brhd1zv3ejPO7cHafYd58otdJMaEk9A93OpIyo3pZYmtrLi8moXrDzJpcGeiw4OsjqMsJiL885ohdAwJZPbCLbreizorWuitbOH6gxyvqmXmBb2sjqJcRGiQHy9OT+BQSQUP6frp6ixoobeiyppa3lyznzGxHRjQJdTqOMqFDOsezoPj+/FFah7v6PrpqoW00FvRp1tyyD9WyW8v1KNz9Uu/GdOLsf0i+euynWzPLrY6jnJDWuitpK7OMHd1OvGdQxgT+4sp+kphswlPTxlKeLAfsxfq/UhV82mht5JVewtIyy/ljgt60jBlX6lfiAj254WpCRwsLOORj7fpeLpqFi30VvLW2gNEtgvg8kFdrI6iXNw5vdpz3yV9WZqcw/tJmWd+gVINtNBbwb6CUr7fXcCN5/TA31c/cnVmd46N5bzY9vzv0lR25x2zOo5yE9ourWDB2gP4+9h0ES5lNx+b8Oz1Q2kb4MvshZspq9LxdHVmWuhOVlJRzYebspg0pDOR7QKsjqPcSFS7QJ67PoG0glL+39JUq+MoN6CF7mTvb8zkeFUtt47Wy/xV843p04G7Lorl/aQslmzJsjqOcnFa6E5UW2dY8GMGiT3CGRStFxKplrn3kj6MiAnn0SXbSS8otTqOcmFa6E707a58DhaWcet5enSuWs7Xx8YL0xII8LVx18ItVFTXWh1JuSgtdCd6c81+OocGctmAjlZHUW6uc2gbnp4yhJ25Jfzf8p1Wx1EuSgvdSXbnHWPtviPcdG4PfH30Y1Znb1xcR+44vyfvrMtgxbZcq+MoF6RN4yTvrsvA39fG1BE6VVE5zgOXxTGkWxgPfZjCwSNlVsdRLkYL3QmOV9awZEs2kwZ1JiJY70ikHMff18ZL0xJA4K6Fm6ms0fF09TMtdCf4ZGs2pZU13KD3C1VO0C0iiH9dO4Rt2cX8TcfTVSNa6A5mjOHddQfp3zmEYd3DrI6jPNT4gZ247byevP1jBstTdDxd1dNCd7AtmUXszC3hxlHddVVF5VQPT2gYT/8ohQOHj1sdR7kALXQHe3ddBsH+Pkwe2tXqKMrD+fvaeHl6Aj424a6Fm3V+utJCd6Sjx6tYlpLLVcO60jbA1+o4ygtEhwfx9HVDSM0p4a/Ld1gdR1lMC92BPtyURVVNHTfqyVDVii6J78jMC3rx7rqDfJacY3UcZSEtdAepqzO8t75+3Za4TiFWx1Fe5oHL+jG8RzgPf5Si6714MS10B1mz7zAHjpTp0bmyhJ+PjRenJeDva+PO93Q83VtpoTvIe+sOEh7kx/iBnayOorxUl7A2PHP9UHblHeMvn+n66d5IC90B8o9V8PXOQ1yX2I1APx+r4ygvNrZfFL+7qDeLNmTyyZZsq+OoVqaF7gAfb86mps5w/YhuVkdRij/+qi8jYyJ4ZMk20vJ1PN2baKGfJWMM/96YyYiYcHpHtrU6jlI/rZ/exs+Hu97bTHmVjqd7Cy30s7RhfyH7Dx/nel1VUbmQTqGBPHv9UPbkH+NPn263Oo5qJVroZ+nfSZm0C/Bl4iA9GapcywV9I5k9NpYPN2WxeMNBq+OoVqCFfhaKy6tZsS2XK4Z2IchfrwxVrufeS/pyfp8O/PnTVJIzi6yOo5xMC/0sLE3OoaK6jql6MlS5KB+b8PzUBCLbBXDne5spPF5ldSTlRFroZ+H9jZn07xzCoK6hVkdR6pQigv159cZhFByr5J7FW6itM1ZHUk6ihd5C27OL2ZZdzNQR3XSZXOXyBkeH8fjkAazee5jnvt5jdRzlJFroLfR+Uib+vjau1GVylZuYOrI71yd248Vv0/h6xyGr4ygn0EJvgYrqWj7Zks2EgZ0IDfKzOo5SdvvL5AEM6hrKfe9v1ZtieCAt9Bb4YnseJRU1emWocjuBfj68csMwfGzCrHc36UVHHsauQheR8SKyW0TSROThkzx/g4ikNPxaKyJDHB/Vdfx7YybdI4IY1bO91VGUarZuEUE8PzWB3YeO8ciSbRijJ0k9xRkLXUR8gJeBCUA8ME1E4ptsth+40BgzGHgCmOvooK4is7CMH9OPcO3waGw2PRmq3NOFfSO575K+LNmSzYIfM6yOoxzEniP0kUCaMSbdGFMFLAYmN97AGLPWGHO04dt1QLRjY7qOJQ0r2F2VoCdDlXubPTaWi+OieGLZDtanH7E6jnIAewq9K5DZ6PushsdO5Xbg85M9ISIzRSRJRJIKCgrsT+kijDF8vDmLc3u1p1tEkNVxlDorNpvw7NShdG8fxJ3vbSbraJnVkdRZsqfQTzaucNJBNxEZS32hP3Sy540xc40xicaYxMjISPtTuohNGUc5cKSMa4Z77A8gysuEBPoxb0YiVTV1zFywibKqGqsjqbNgT6FnAY2nc0QDv7gTrYgMBl4HJhtjPPLntw83ZRHk78MEvSuR8iC9I9vywrQEduaV8MCHKXqS1I3ZU+gbgT4i0lNE/IGpwNLGG4hId+Bj4CZjjEdehlZRXcvylFzGD+xEcIAuxKU8y9i4KB4aH8fylFxe+X6f1XFUC52xmYwxNSIyG1gJ+ADzjTGpIjKr4fk5wJ+B9sArDZfB1xhjEp0Xu/WtTM3jWGUN1w7T4RblmX57QS925JTw1Je76dexHZfEd7Q6kmomserHq8TERJOUlGTJe7fEjPkb2JdfyuoHx+p0ReWxyqtque61tRw4XMYnd40mNqqd1ZFUEyKy6VQHzHqlqB3yiiv4z94Crh7WVctcebQ2/j7MvSmRQD8bdyzYRHFZtdWRVDNoodthyZZs6gxcrcMtygt0CWvDqzcOJ+toGbMXbaamts7qSMpOWuhnYIzho81ZDO8RTs8OwVbHUapVjIiJ4InJA1m99zB/+WyHznxxE1roZ5CSVUxafinX6NG58jJTR3bntxf04p11Gby19oDVcZQddP7dGXy0OYsAXxuXD+5sdRSlWt1D4+PYf/g4TyzbQfeIIC7urzNfXJkeoZ9GZU0tn27N4dIBnQhto+ueK+9jswnPTR3KgC6h/H7RFnbklFgdSZ2GFvppfLergOLyaq4epgtxKe8V5O/L6zcnEtrGj9vf3sihkgqrI6lT0EI/jU+2ZNOhrT/nx3awOopSluoYEsgbN4+gpLya37ydpGu+uCgt9FMoLq/m2135TBrcBV8f/ZiUiu8SwovTE0jNKebexVupq9OZL65Gm+oUvtieS1VtHVfquudK/WRcXEf+NCmeL3cc4vFlOp3R1egsl1P4ZEsOMe2DGBIdanUUpVzKLaNjyCwsZ/6a/XQODeS3F/a2OpJqoIV+ErnF5azbf4R7Lu5Dw2JjSqkGIsJjl/cn/1gFf/98F1EhAVyVoNdpuAIt9JNYujUHY+DKoTrcotTJ2GzC01OGcKS0igc+SKFD2wDO7+N+N63xNDqGfhKfbM1hSLcwYvRSf6VOKcDXh9dmDCc2qi2z3tnE9uxiqyN5PS30JnbnHWNnbglXDe1idRSlXF5IoB9v3zaSsCB/bnlzIweP6H1JraSF3sQnW7PxsQmThmihK2WPjiGBvH3bCKpr67j5zQ0cKa20OpLX0kJvpK7OsHRrDmNiO9ChbYDVcZRyG7FR7Zh/SyI5ReXMmL+BkgpdR90KWuiNJGUcJbuonCsT9OhcqeYa3iOCOTcOZ3feMW5/ayPlVbVWR/I6WuiNLNmSTRs/Hy6N72R1FKXc0ti4KJ6fmsCmjKPMfCeJyhot9dakhd6gqqaOFdtyuXRAR4IDdDanUi11+eDO/OPqwazee5h7Fm3VOx61Ii30Bt/vzqe4vFrnnivlAFNGdONPk+L5IjWPhz7apuu+tBI9FG3w6dYcIoL9GdNHV1ZUyhFuH9OT0ooanv16D+0CffnfK+L1ymsn00IHSiqq+WrnIaaN6IafrqyolMPcfXEspZXVzFu9H39fG/8zIU5L3Ym00IGV2/Ooqqljsq6sqJRDiQiPTOxPZU0dc1elA2ipO5EWOvBZSi7R4W1I6BZmdRSlPI6I8JdfDwBg7qp0BHhYS90pvL7Qj5RWsibtMDMv6KU7mFJOcqLUjYHXVqWDwMPjtdQdzesL/YvUPGrrDFcM1ouJlHImEeHxyQMwGF77oX74RUvdsby+0D9LzqF3ZDD9O7ezOopSHk9EeGLyQABe+yEdY3RM3ZG8utAPlVSwfn+h3shCqVYkIjz+64EIwtxV6RyvrOGJyQOx2fTv4Nny6kJfnpKLMTBJh1uUalU2W/3wS1CAD6/9kE5ZVS3/unaw3pD9LHl1oX+WkkN85xBio9paHUUpryMiPDw+jpBAP/61cjdlVTW8MC2BAF8fq6O5La/95zCzsIwtB4u4Qtc9V8oyIsJdY2P5f1fEszL1EL95O4myqhqrY7ktry30ZSm5AEwa3NniJEqpW87ryT+vHcyatMPMeGMDxWW6nnpLeG2hf5acQ0L3MLpFBFkdRSkFTEnsxovThpGcVcR1r60lp6jc6khuxysLPS2/lB25JTr3XCkXc/ngzrx960hyiyq46pU17MwtsTqSW/HKQl+WkoNI/c6jlHIto2M78P6scwGYMudH1u47bHEi9+F1hW6M4bPkHM7pGUHHkECr4yilTqJ/5xA+vvM8OoUGcvP8DSxNzrE6kluwq9BFZLyI7BaRNBF5+CTPx4nIjyJSKSL3Oz6m4+zMPca+guM691wpF9c1rA0fzhpNQrdw7l60hTk/7MMYvVHG6Zyx0EXEB3gZmADEA9NEJL7JZoXA3cBTDk/oYJ+l5OBjEyYM1PuGKuXqQoP8WHD7yPrb2n2+i/s/SNH7lJ6GPUfoI4E0Y0y6MaYKWAxMbryBMSbfGLMRcOm5RieGW86L7UD7tgFWx1FK2SHQz4cXpyZwz8V9+GhzFjfMW8/h0kqrY7kkewq9K5DZ6PushseaTURmikiSiCQVFBS05Lc4K1szi8g6Ws4VejJUKbdiswn3/aovL01PYFt2MZNfWsOuPJ0B05Q9hX6yFXNaNJBljJlrjEk0xiRGRka25Lc4K58l5+LvY+PSATrcopQ7mjS4Cx/MOpeaujqueWUtX+04ZHUkl2JPoWcB3Rp9Hw243Snn2jrDspQcLuwXSWgbP6vjKKVaaHB0GJ/eNYbeUW25Y0EST63cTW2dniwF+wp9I9BHRHqKiD8wFVjq3FiOt/FAIfnHKnXtFqU8QKfQQN7/7blMSYzmpe/SuOXNDRQer7I6luXOWOjGmBpgNrAS2Am8b4xJFZFZIjILQEQ6iUgW8AfgMRHJEpEQZwZvrs+Scwj0s3FJ/yiroyilHCDQz4d/XjuEf1w9iPX7C7nixf+QnFlkdSxL2TUP3RizwhjT1xjT2xjzfw2PzTHGzGn4Os8YE22MCTHGhDV87TJnLGrrDCtT87g4riNB/l69YrBSHmfqyO582HBl6XVzfuS99RleO1/dK64UXb//CIdLq/RSf6U81ODoMJb9fgyjerfn0SXbmb1wC8XlLj2L2im8otCXp+TSxs+Hsf10uEUpTxUe7M9bt4zgofFxrEzNY+Lzq0k6UGh1rFbl8YV+YrhlXFwUbfz1TihKeTKbTfjdRb358Hej8bEJU177kRe+2es1s2A8vtBPDLdMHKTDLUp5i6Hdwlh+9xh+PaQLz3y1h2nz1pFZWGZ1LKfz+EJfsS2XQD8bY+Na/0ImpZR12gX68dzUBJ6ZMoQdOSWMf26Vx58w9ehCr60zfLFdZ7co5c2uHhbNF/eez9DuYTy6ZDsz5m/w2LsheXSh63CLUgogOjyId28/hyeuHMimjKNc9uwq3t+Y6XFH6x5d6DrcopQ6QUS4aVQPvrjnAuK7hPDgRync+MZ69h8+bnU0h/HYQq8fbjnEuLgoHW5RSv2ke/sgFt0xiicmDyAls5jLnlvFC9/s9Yh11j220DfsL+RwaSWXD9K1W5RS/81mE246N4Zv/nghl8Z35Jmv9jDh+dWsSz9idbSz4rGFvnxbjg63KKVOKyokkJemD+OtW0dQXVvH1LnruO/fW8krrrA6Wot4ZKHrcItSqjku6hfFl/deyOyxsSzflsvYp77n+a/3Ul7lXsMwHlnoJ4ZbdHaLUspebfx9uP+yfnzzhwsZGxfJs1/v4eKnv2dpco7bzIbxyEI/MbtlXJyu3aKUap5uEUG8csNwFs8cRViQP3cv2sLVr65l7b7DVkc7I48r9No6w+fb83S4RSl1Vkb1as9nvx/Dk9cMIreogunz1nPTG+tJySqyOtopeVyh63CLUspRfGzC9SO68/0DF/HY5f3Znl3Mr19aw6x3NpGWf8zqeL/gcYewOtyilHK0QD8ffnN+L64f0Y3XV+/n9dXpfLkjj4mDOnPX2Fj6d3aNG7R5VKGfGG4Z20+HW5RSjtcu0I/7ftWXm0fHMHdVOu+uy2BZSi4Xx0Vx17hYhnUPtzSfRw25bDygwy1KKeeLCPbn4QlxrHloHH/4VV82HTzK1a+sZfq8dazaU2DZrBiPKvTlKTrcopRqPaFBftx9cR/WPDSOxy7vT1p+KTPmb+BXz9Yv1dva89g9ptAbD7cEB+hwi1Kq9QQH+PKb83ux+qGxPDNlCIF+Nh5dsp1Rf/+Gv3++k+xWWq7XY5pPh1uUUlYL8PXh6mHRXJXQlaSMo7y5Zj/zVqUzb1U64+I6Mv2cblzYNwofmzjl/T2m0FdsyyXAV4dblFLWExFGxEQwIiaCrKNlvLf+IB8kZfL1zkN0DWvDPZf0YUpiN4e/r0cUeuOLiXS4RSnlSqLDg3hofBz3XdKXr3ceYtGGg067abVHtN/GA4UUHNPhFqWU6/L3tTFxUGcmDurstFkwHnFSVIdblFLuRMQ5Y+huX+g6u0Uppeq5faEnnRhuGazDLUop7+b2hb68YbjlYh1uUUp5ObcudB1uUUqpn7l1oetwi1JK/cytC32FDrcopdRP3LbQTwy3XNQvUodblFIKNy70pAOF5B+r5PLBXayOopRSLsFtC12HW5RS6r+5ZaHX6XCLUkr9glsWelLGUfJ17RallPovdhW6iIwXkd0ikiYiD5/keRGRFxqeTxGRYY6P+rPlKTn1wy39OzrzbZRSyq2csdBFxAd4GZgAxAPTRCS+yWYTgD4Nv2YCrzo4508aD7e01eEWpZT6iT1H6COBNGNMujGmClgMTG6yzWRggam3DggTEaeMh+hwi1JKnZw9hd4VyGz0fVbDY83dBhGZKSJJIpJUUFDQ3KwA2AQu7Bupwy1KKdWEPYV+soV7m67Obs82GGPmGmMSjTGJkZGR9uT7hcSYCN6+baQOtyilVBP2FHoW0Pjmd9FATgu2UUop5UT2FPpGoI+I9BQRf2AqsLTJNkuBGQ2zXUYBxcaYXAdnVUopdRpnHLcwxtSIyGxgJeADzDfGpIrIrIbn5wArgIlAGlAG3Oq8yEoppU7GroFoY8wK6ku78WNzGn1tgLscG00ppVRzuOWVokoppX5JC10ppTyEFrpSSnkILXSllPIQUn8+04I3FikAMlr48g7AYQfGcRRXzQWum01zNY/mah5PzNXDGHPSKzMtK/SzISJJxphEq3M05aq5wHWzaa7m0VzN4225dMhFKaU8hBa6Ukp5CHct9LlWBzgFV80FrptNczWP5moer8rllmPoSimlfsldj9CVUko1oYWulFIewuUK/WxuSH2m1zo51w0NeVJEZK2IDGn03AER2SYiW0UkqZVzXSQixQ3vvVVE/mzva52c64FGmbaLSK2IRDQ858zPa76I5IvI9lM8b9X+daZcVu1fZ8pl1f51plytvn+JSDcR+U5EdopIqojcc5JtnLt/GWNc5hf1y/PuA3oB/kAyEN9km4nA59TfJWkUsN7e1zo512ggvOHrCSdyNXx/AOhg0ed1EbCsJa91Zq4m218BfOvsz6vh974AGAZsP8Xzrb5/2Zmr1fcvO3O1+v5lTy4r9i+gMzCs4et2wJ7W7i9XO0I/mxtS2/Nap+Uyxqw1xhxt+HYd9Xdtcraz+X+29PNqYhqwyEHvfVrGmFVA4Wk2sWL/OmMui/Yvez6vU7H082qiVfYvY0yuMWZzw9fHgJ388t7KTt2/XK3Qz+aG1HbdqNqJuRq7nfp/hU8wwJcisklEZjooU3NynSsiySLyuYgMaOZrnZkLEQkCxgMfNXrYWZ+XPazYv5qrtfYve7X2/mU3q/YvEYkBEoD1TZ5y6v7landaPpsbUtt1o+oWsvv3FpGx1P+FG9Po4fOMMTkiEgV8JSK7Go4wWiPXZurXfigVkYnAJ0AfO1/rzFwnXAGsMcY0Ptpy1udlDyv2L7u18v5lDyv2r+Zo9f1LRNpS/w/IvcaYkqZPn+QlDtu/XO0I/WxuSO3MG1Xb9XuLyGDgdWCyMebIiceNMTkN/80HllD/41Wr5DLGlBhjShu+XgH4iUgHe17rzFyNTKXJj8NO/LzsYcX+ZRcL9q8zsmj/ao5W3b9ExI/6Mn/PGPPxSTZx7v7l6BMDZ/OL+p8Y0oGe/HxiYECTbS7nv08qbLD3tU7O1Z36e6qObvJ4MNCu0ddrgfGtmKsTP19ANhI42PDZWfp5NWwXSv04aHBrfF6N3iOGU5/ka/X9y85crb5/2Zmr1fcve3JZsX81/H8vAJ47zTZO3b8c9uE68A9pIvVnh/cBjzY8NguY1ehDe7nh+W1A4ule24q5XgeOAlsbfiU1PN6r4Q8nGUi1INfshvdNpv5k2ujTvba1cjV8fwuwuMnrnP15LQJygWrqj4pud5H960y5rNq/zpTLqv3rtLms2L+oHwYzQEqjP6eJrbl/6aX/SinlIVxtDF0ppVQLaaErpZSH0EJXSikPoYWulFIeQgtdKaU8hBa6Ukp5CC10pZTyEP8fC7OtFlrn9uQAAAAASUVORK5CYII=\n",
       "text/plain": [
        "<AxesSubplot:>"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "plt.plot(X, Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_features(X):\n",
    "    f1 = tf.ones_like(X)  # Bias.\n",
    "    f2 = X\n",
    "    f3 = tf.square(X)\n",
    "    f4 = tf.sqrt(X)\n",
    "    f5 = tf.exp(X)\n",
    "    return tf.stack([f1, f2, f3, f4, f5], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(X, W):\n",
    "    return tf.squeeze(X @ W, -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_mse(X, Y, W):\n",
    "    Y_hat = predict(X, W)\n",
    "    errors = (Y_hat - Y)**2\n",
    "    return tf.reduce_mean(errors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_gradients(X, Y, W):\n",
    "    with tf.GradientTape() as tape:\n",
    "        loss = loss_mse(Xf, Y, W)\n",
    "    return tape.gradient(loss, W)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD6CAYAAACoCZCsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjUUlEQVR4nO3deXhc1X3G8e9Po30dWZJlW5ItG2wwYMvYigVhK8EkLAkmS1NIWgdI6jgJBZImKWnShjZNm6ZZGgrFNQlNoCQQCElc4oQAZQmbsQ3G+yJvWLIsW7K179LpH3PtjIWWkTXSlTTv53nmmZl7z9H85kqad+49dzHnHCIiEnvi/C5ARET8oQAQEYlRCgARkRilABARiVEKABGRGKUAEBGJUREFgJldZWY7zazczO7sY76Z2d3e/E1mtnCwvma2wMxeM7ONZrbezBZH5y2JiEgkbLDjAMwsAOwCrgQqgHXAjc65bWFtrgH+CrgGKAN+4JwrG6ivmf0e+L5z7rde/y875/5koFpyc3NdcXHxab1REZFYtWHDhhrnXF7v6fER9F0MlDvn9gKY2SPAUmBbWJulwIMulCavmVnQzKYCxQP0dUCm1z8LODRYIcXFxaxfvz6CkkVE5AQzO9DX9EgCoAA4GPa8gtC3/MHaFAzS9w7gKTP7DqFNUe/up/DlwHKA6dOnR1CuiIhEIpIxAOtjWu/tRv21GajvZ4DPO+eKgM8DP+rrxZ1zq5xzpc650ry8d6zBiIjIaYokACqAorDnhbxzc01/bQbq+wngCe/xY4Q2NYmIyCiJJADWAbPNbKaZJQI3AKt7tVkNLPP2BroAqHfOVQ3S9xBwmff4PcDuYb4XEREZgkHHAJxzXWZ2K/AUEAAecM5tNbMV3vyVwBpCewCVAy3AzQP19X70XwI/MLN4oA1vO7+IiIyOQXcDHUtKS0ud9gISERkaM9vgnCvtPV1HAouIxKiYCIDndhzhP58v97sMEZExJSYC4JU9Nfz7M7vp6OrxuxQRkTEjJgKgpChIR1cPOw83+l2KiMiYERsBUBgEYGNFna91iIiMJTERAIXZKeSkJfLWwTq/SxERGTNiIgDMjJKioAJARCRMTAQAhDYDlR9torGt0+9SRETGhJgJgAXTgzgHmyvr/S5FRGRMiJkAKCnMAmCjNgOJiAAxFADB1ESKc1I1DiAi4omZAAC8gWBtAhIRgVgLgMIghxvaOFzf5ncpIiK+i60AKAoC8JYOCBMRia0AOHdaJvFxpnEAERFiLACSEwLMnZqpPYFERIixAAAoKcpiU0U9PT3j50I4IiIjIfYCoDBIU3sXe2ua/C5FRMRXMRcAC7yB4I3aHVREYlzMBcCsvHTSk+I1ECwiMS/mAiAQZ8wryNKuoCIS82IuACB0YrjtVQ20dXb7XYqIiG9iMgBKCoN0dju2VTX4XYqIiG9iMgBODARrHEBEYllMBsCUrGTyM5MUACIS02IyACC0GeitCu0KKiKxK3YDoCjIvppm6lo6/C5FRMQXMRsA53vjAJu0FiAiMSqiADCzq8xsp5mVm9mdfcw3M7vbm7/JzBYO1tfMHjWzjd5tv5ltjMo7itB5hVmY6RKRIhK74gdrYGYB4F7gSqACWGdmq51z28KaXQ3M9m5lwH1A2UB9nXN/FvYa3wVG9at4ZnICZ+SlayBYRGJWJGsAi4Fy59xe51wH8AiwtFebpcCDLuQ1IGhmUyPpa2YGfBT42TDfy5CFBoLrcE5nBhWR2BNJABQAB8OeV3jTImkTSd9LgGrn3O6+XtzMlpvZejNbf/To0QjKjdyCoixqmjqorGuN6s8VERkPIgkA62Na76/M/bWJpO+NDPDt3zm3yjlX6pwrzcvLG7DQoTp5iUidGVREYlAkAVABFIU9LwQORdhmwL5mFg98CHg08pKj5+wpmSTGx+nEcCISkyIJgHXAbDObaWaJwA3A6l5tVgPLvL2BLgDqnXNVEfRdAuxwzlUM+52chsT4OM6dpktEikhsGnQvIOdcl5ndCjwFBIAHnHNbzWyFN38lsAa4BigHWoCbB+ob9uNvwIfB33AlhUEeXXeQru4e4gMxe1iEiMSgQQMAwDm3htCHfPi0lWGPHfC5SPuGzbsp0kJHyoKiID9+ZT+7jzQxd2qm3+WIiIyamP/KW6Izg4pIjIr5ACjOSSUzOV4DwSISc2I+AMyMkqKgLhIvIjEn5gMAQuMAu6obaeno8rsUEZFRowAgFADdPY4tlbpEpIjEDgUAML8wCGggWERiiwIAyMtIoiCYwkYNBItIDFEAeBYUBbUGICIxRQHgKSnKouJ4KzVN7X6XIiIyKhQAnhJvHGCTNgOJSIxQAHjmFWYRZ+h4ABGJGQoAT2piPHPyM3RmUBGJGQqAMCcGgnWJSBGJBQqAMCVFQepbOzlQ2+J3KSIiI04BEObEQLBODCcisUABEGZOfjrJCXEaBxCRmKAACBMfiGNeQZYOCBORmKAA6GVBUZAthxro7O7xuxQRkRGlAOilpChIR1cPO6oa/S5FRGREKQB6OTEQrBPDichEpwDopTA7hZy0RI0DiMiEpwDo5cQlIhUAIjLRKQD6UFIYpPxoE41tnX6XIiIyYhQAfVgwPYhzsLlSJ4YTkYlLAdCHksIsAB0QJiITmgKgD8HURIpzUjUOICITmgKgH6GBYG0CEpGJK6IAMLOrzGynmZWb2Z19zDczu9ubv8nMFkbS18z+ypu31cy+Pfy3Ez0lhUEON7RxuL7N71JEREZE/GANzCwA3AtcCVQA68xstXNuW1izq4HZ3q0MuA8oG6ivmV0OLAXmO+fazWxyNN/YcJUUBYHQmUGnZE3xtxgRkREQyRrAYqDcObfXOdcBPELogzvcUuBBF/IaEDSzqYP0/QzwLedcO4Bz7kgU3k/UnDstk/g40ziAiExYkQRAAXAw7HmFNy2SNgP1nQNcYmZrzewFM3tXXy9uZsvNbL2ZrT969GgE5UZHckKAuVMzdW0AEZmwIgkA62Na72sm9tdmoL7xQDZwAfAl4Odm9o72zrlVzrlS51xpXl5eBOVGT0lRFpsO1tPTo0tEisjEE0kAVABFYc8LgUMRthmobwXwhLfZ6HWgB8iNvPSRV1IYpLG9i701TX6XIiISdZEEwDpgtpnNNLNE4AZgda82q4Fl3t5AFwD1zrmqQfr+CngPgJnNARKBmuG+oWha4A0Eb9TuoCIyAQ0aAM65LuBW4ClgO/Bz59xWM1thZiu8ZmuAvUA5cD/w2YH6en0eAGaZ2RZCg8OfcM6NqW0ts/LSSU+K10CwiExIg+4GCuCcW0PoQz582sqwxw74XKR9vekdwJ8PpdjRFoiz0CUiNRAsIhOQjgQexILpQbZXNdDW2e13KSIiUaUAGERJYZDObse2qga/SxERiSoFwCBODARrHEBEJhoFwCCmZCWTn5mkABCRCUcBEIGSwiAb3j6uA8JEZEJRAETg6nlTOHislcc3VPhdiohI1CgAIrC0pIDSGdn8y2+3c7y5w+9yRESiQgEQgbg44xvXn0dDWxfffmqn3+WIiESFAiBCc6dmctO7i3lk3du6VrCITAgKgCG4Y8ls8tKT+NqvNtOtAWERGecUAEOQkZzA373/HLZUNvDw2gN+lyMiMiwKgCF6//ypXHxmLv/21E6ONrb7XY6IyGlTAAyRmfEPS8+lrbObf/ntdr/LERE5bQqA03BGXjrLL53FE29UsnZvrd/liIicFgXAabr18tkUBFP4u19vobO7x+9yRESGTAFwmlISA9x13bnsqm7ixy/v97scEZEhUwAMw5Xn5HPF2ZP5/jO7qKpv9bscEZEhUQAM013XnUt3j+OfntSAsIiMLwqAYSqalMqtl5/JbzZX8eKuo36XIyISMQVAFCy/bBYzc9P4+uqttHfp0pEiMj4oAKIgKT7AP1x3Lvtqmln1wl6/yxERiYgCIEounZPHtfOmcs9z5Rw81uJ3OSIig1IARNHX3j+XQJxx1+qtfpciIjIoBUAUTc1K4fNL5vDsjiM8va3a73JERAakAIiymy4qZk5+Onet3kprhwaERWTsUgBEWUIgjm8sPY/KulbueW633+WIiPRLATACymbl8KGFBax6cS97jjb5XY6ISJ8UACPkK1fPJTkhwN//egvO6ephIjL2RBQAZnaVme00s3Izu7OP+WZmd3vzN5nZwsH6mtldZlZpZhu92zXReUtjQ15GEl9631m8XF7Lk5uq/C5HROQdBg0AMwsA9wJXA+cAN5rZOb2aXQ3M9m7Lgfsi7Pt959wC77ZmuG9mrPl42QzOK8jkG09uo7Gt0+9yREROEckawGKg3Dm31znXATwCLO3VZinwoAt5DQia2dQI+05YgTjjn66fx9Gmdv79GQ0Ii8jYEkkAFAAHw55XeNMiaTNY31u9TUYPmFl2Xy9uZsvNbL2ZrT96dPydbG1BUZAbF0/ngZf38X87dGyAiIwdkQSA9TGt96hmf20G6nsfcAawAKgCvtvXizvnVjnnSp1zpXl5eRGUO/Z87dq5nDM1k9t/tpHyI41+lyMiAkQWABVAUdjzQuBQhG367eucq3bOdTvneoD7CW0umpBSE+NZtayUpIQ4PvWT9dS3aDxARPwXSQCsA2ab2UwzSwRuAFb3arMaWObtDXQBUO+cqxqorzdGcMIHgS3DfC9jWkEwhZV/vojKulZu/dkbdOk6wiLis0EDwDnXBdwKPAVsB37unNtqZivMbIXXbA2wFygn9G3+swP19fp828w2m9km4HLg89F7W2NTafEkvnn9PP6wu4Z/XrPD73JEJMbZeDpIqbS01K1fv97vMobtrtVb+fEr+/n2R+bz0dKiwTuIiAyDmW1wzpX2nq4jgX3wtWvncvGZuXztl1vYcOCY3+WISIxSAPggPhDHPR87n6nBZD790Bscqmv1uyQRiUEKAJ8EUxP54bJS2jq7Wf7Qep06WkRGnQLAR7PzM/jBDQvYeqiBL/9ik04aJyKjSgHgsyvm5vOl953F/751iP98fo/f5YhIDFEAjAGfuewMriuZxnd+v5NndClJERklCoAxwMz49kfmc960LG5/5E12Vet0ESIy8hQAY0RyQoBVyxaRkhjPp36ynuPNHX6XJCITnAJgDJmalcJ//cUiDte38bmfvkGnThchIiNIATDGLJqRzT9/aB6v7Knlm7/Z7nc5IjKBxftdgLzTRxYVsqOqgR++tI+zp2Rww+LpfpckIhOQ1gDGqDuvPptLZufyd7/ewrr9Ol2EiESfAmCMig/Ecc+NCynMTmXFQxuo1OkiRCTKFABjWFZqAvcvK6Wjq4dP/nid9gwSkahSAIxxZ05O596PL2RvTTM3rHqNI41tfpckIhOEAmAcuHROHv9907s4eLyFj658VZuDRCQqFADjxEVn5vLQJxdT29zBR1e+yr6aZr9LEpFxTgEwjiyaMYmf/eUFtHZ289H/epWdh3XKCBE5fQqAcea8giweXX4BBvzZqlfZXFHvd0kiMk4pAMah2fkZPLbiQtIS4/nY/a/pOAEROS0KgHFqRk4aj624kLyMJJb96HVe2l3jd0kiMs4oAMaxacEUHv30hczISeWWH6/jaV1LQESGQAEwzuVlJPHI8guYOy2TFf+zgV9vrPS7JBEZJxQAE0AwNZGHP1XGohnZ3PHoRh55/W2/SxKRcUABMEGkJ8Xzk5sXc+nsPO58YjM/emmf3yWJyBinAJhAUhJDVxV737n5fOPJbdzzf7txzvldloiMUQqACSYpPsC9H1vIB88v4Du/38W//m6nQkBE+hRRAJjZVWa208zKzezOPuabmd3tzd9kZguH0PeLZubMLHd4b0VOiA/E8d0/LeFjZdNZ+cIevr56Kz09CgEROdWgVwQzswBwL3AlUAGsM7PVzrltYc2uBmZ7tzLgPqBssL5mVuTN06hllMXFGd+8/jzSEgPc/4d9NLV38c8fnEdyQsDv0kRkjIhkDWAxUO6c2+uc6wAeAZb2arMUeNCFvAYEzWxqBH2/D3wZ0NfTEWBm/O01c/n8kjk88UYlH77vFZ1ETkROiiQACoCDYc8rvGmRtOm3r5ldB1Q6594aYs0yBGbG7Utm88NlpVTWtfKB/3iJ1W8d8rssERkDIgkA62Na72/s/bXpc7qZpQJfBf5+0Bc3W25m681s/dGjRwctVvq25Jx8fnPbJZw1JYPbfvYmf/vLzbR1dvtdloj4KJIAqACKwp4XAr2/QvbXpr/pZwAzgbfMbL83/Q0zm9L7xZ1zq5xzpc650ry8vAjKlf4UBFN4ZPkFfPqyWfx07dtcf+/L7Dna5HdZIuKTSAJgHTDbzGaaWSJwA7C6V5vVwDJvb6ALgHrnXFV/fZ1zm51zk51zxc65YkJBsdA5dzhab0z6lhCI4ytXz+W/b3oX1Q1tfOA/XuJXb+r0ESKxaNAAcM51AbcCTwHbgZ8757aa2QozW+E1WwPsBcqB+4HPDtQ36u9Chuzysyez5vZLOHdaJnc8upG/eXwTrR3aJCQSS2w8HSRUWlrq1q9f73cZE0pXdw/fe3oX//n8Hs7Kz+Dej5/PmZMz/C5LRKLIzDY450p7T9eRwDEuPhDHl686m5/cspiapnY+8B8v8/iGCr/LEpFRoAAQAC6bk8ea2y9hfmEWX3zsLf7652/R0tHld1kiMoIUAHJSfmYyD3+qjNvecyZPvFnBdfe8rAvPi0xgCgA5RXwgji+89yweuqWMupZOlt77Eo+ue1snlBOZgBQA0qeLZ+ey5vaLWTg9m7/5xWZuf2QjNU3tfpclIlGkAJB+Tc5I5qFPlvGFK+ewZnMVl3/neR54aR+d3T1+lyYiUaAAkAEF4ozbrpjN7+64lAVFQf7xyW1ce/cfeKW8xu/SRGSYFAASkTMnp/PgLYtZ9ReLaO3s5mM/XMtnH95AxfEWv0sTkdOkAJCImRnvPXcKT3/+Mr5w5Rz+b8cRlnzvBX7wzG6dWE5kHFIAyJAlJwS47YrZPPvXf8IVZ+fz/Wd2seR7L/C7LYe1t5DIOKIAkNNWEEzh3o8v5KefKiM1McCK/9nAsgdep/yIjh0QGQ8UADJs7z4zlzW3XcLXP3AOGw/WcdW//4F/enIbjW2dfpcmIgNQAEhUxAfiuPmimTz3xT/hI4sK+dHL+7j8Oy/w2PqDuiC9yBilAJCoyk1P4lsfns+vPnsRhdkpfOnxTXzovlfYeLDO79JEpBcFgIyIkqIgT3zm3XznT0uoON7K9fe+zCceeJ1X99RqoFhkjND1AGTENbZ18uCrB/jvl/dR09TBgqIgKy47g/eek09cXF+XjRaRaOrvegAKABk1bZ3dPLahgvtf3Mvbx1o4Iy+NT196BtefX0BivFZGRUaKAkDGjK7uHtZsOczK5/ewraqBKZnJfPLimdxYNp30pHi/yxOZcBQAMuY453hxdw0rn9/Dq3tryUyOZ9mFxdx0UTG56Ul+lycyYSgAZEzbeLCOlc/v4alth0kMxPGnpYUsv+QMpuek+l2ayLinAJBxYc/RJla9sJcn3qygu8dx7fxprLhsFudOy/K7NJFxSwEg48rh+jYeeHkfD792gOaObi6ZncufvauIJXPzSU4I+F2eyLiiAJBxqb6lk/9Ze4CHXj3A4YY2MpPj+UDJND68qJDzi4KYaTdSkcEoAGRc6+5xvLKnhsc3VPDU1sO0dfYwKy+NDy8s5EMLC5ialeJ3iSJjlgJAJozGtk7WbK7i8Q0VrNt/HDO4+MxcPrKokPeeM4WURG0iEgmnAJAJ6UBtM794o5JfbKigsq6V9KR43j9/Kh9eVEjpjGxtIhJBASATXE+PY+2+Yzy+oYLfbqmipaObGTmpJzcRFWZrd1KJXQoAiRnN7V38bsthHt9Qwat7awEomzmJK8/JZ8ncfIpz03yuUGR0DSsAzOwq4AdAAPihc+5bveabN/8aoAW4yTn3xkB9zewbwFKgBzji9Tk0UB0KABmqiuMt/PKNSv530yF2VTcBcEZeGkvm5nPF3HwWTg8SH9B5iGRiO+0AMLMAsAu4EqgA1gE3Oue2hbW5BvgrQgFQBvzAOVc2UF8zy3TONXj9bwPOcc6tGKgWBYAMx9u1LTy7o5pntx9h7b5aOrsdwdQELj9rMlfMncylc/LITE7wu0yRqOsvACI589ZioNw5t9f7QY8Q+ua+LazNUuBBF0qT18wsaGZTgeL++p748PekAeNnW5SMS9NzUrn5opncfNFMGts6eXFXDc9ur+a5nUf45ZuVxMcZZbMmccXZoU1FOg2FTHSRBEABcDDseQWhb/mDtSkYrK+ZfRNYBtQDl/f14ma2HFgOMH369AjKFRlcRnIC186fyrXzp9Ld43jz7eM8s/0Iz26v5h+f3MY/PrmN2ZPTec/cySyZm8/C6dkEdO0CmWAiCYC+/up7f1vvr82AfZ1zXwW+amZfAW4Fvv6Oxs6tAlZBaBNQBPWKDEkgzigtnkRp8STuvPps3q5t4Znt1Ty7o5of/WEf//XCXoKpCbyreBJlMydxwawc5k7NVCDIuBdJAFQARWHPC4Heg7X9tUmMoC/AT4Hf0EcAiIy26Tmp3HLxTG65eCYNbZ38YVcNL+w6wtp9x3h6WzUAGUnxlBZnUzYrh8UzJzGvIIsEDSbLOBNJAKwDZpvZTKASuAH4WK82q4FbvW38ZUC9c67KzI7219fMZjvndnv9rwN2DPvdiERZZtimIgidpG7tvlrW7jvG2r21PLfzKAApCQEWzcimbOYkymblUFKURVK8jkiWsW3QAHDOdZnZrcBThHblfMA5t9XMVnjzVwJrCO0BVE5oN9CbB+rr/ehvmdlZhHYDPQAMuAeQyFgwJSuZpQsKWLqgAICapnZe98Jg7b5jfPfpXQAkxsdxflGQslk5lM2cxMLp2TpFhYw5OhBMJIrqWjp4fd+xUCjsO8bWQ/X0uNA4w5z8DOYVZDKvMMj8gizOmpKhU1vLqNCRwCI+aGjrZMOB46zff4zNlQ1srqjjeEsnAPFxxllTMphXkMW8wizmFwSZMyVdm44k6hQAImOAc47KulY2V9SzqbKeLZX1bKqop741FAoJAePsKZmcV5DF/MIs5hVkMSc/g8R4DTDL6VMAiIxRzjkqjreyqaKeTZV1J0Ohsa0LgMRAHGdNyWBOfgZnTUlndn7o8bSsZJ3tVCKiABAZR5xzvH2shU0VobWELYfq2VXdxNHG9pNt0pPiOXNyOnPy05mTn+EFQzpTMhUMcioFgMgEcLy5g91HmthV3Xjytru6idrmjpNtMpLjmT351FCYk5/B5IwkBUOMUgCITGC1Te3sqm5i95ETwdDE7urGkwPOAKmJAWbkpFGck0pxrnefk0ZxbprCYYIbzsngRGSMy0lP4sL0JC48I+fkNOccNU0d7K5uZPeRJvbXNrO/ppmdhxt5els1XT1//PKXkhBgRk4qM3PTmJGTxszcVO9e4TCRKQBEJigzIy8jibyMJN59Zu4p87q6ezhU18a+2mYO1Dazr6aZA7Ut7DzcyDPbq+nsfmc4FGanUJidSkEwhYLsFAqzUygIpjApLVEBMU4pAERiUHwgjuk5qd4pr/NOmdfV3UNVfZsXCs3sq2nhQG0zFcdbeW3vMZrau05pn5IQOCUQCsKCoig7hdz0JOJ04rwxSQEgIqeID8RRNCmVoknvDAfnHA2tXVTUtVB5vJWK461U1rVScbyFyrpW3jpYd8q4A4R2Y50WTGZKVjJTMpOZkpXClMyk0H1WMlOzkslNT9LZVX2gABCRiJkZWakJZKVmce60rD7bNLd3UVnX6gVECxXe4+qGNtYfOE51Q9Upm5ggdKqMvPSksJBIPhkO+ZmhaZMzk0hN1EdWNGlpikhUpSXFM8c7WK0vPT2OYy0dHK5vC90a/nhf3dBG+dEmXi6vobHXpiaAtMQAkzOTyUtPOjm+EX6b7N3npGmNIhIKABEZVXFxRm56ErnpSZxX0PdaBEBTe9cpIXG0sT10a2rnSEMb2w838OLu9pNHTJ/yGgaT0v4YCHkZSeSkJ5KbFrrPSU8iJy2R3PQkJqUlxuypNhQAIjImnTjS+czJ6QO2a+3opqapnSMnAqKx7WRQHG0MTd9V3UhtUwcd3T19/ozM5Hhy071w6BUSJ6ZNSkskOy2B7NTECXPxHwWAiIxrKYmBsEHr/jnnaGzv4lhTB7XN7dQ0dVDb1EFtUzu1zR3UNLVT29TB3pom1u3v4FhLB/0dJ5uRHB8KhNTEsPsEstMSmZSaGLoPm5+VkjAmN0kpAEQkJpgZmckJZCYnUJybNmj77h7H8ZY/hsSxlg6ON3dwrLmT4y0dHGvu4HhLB9UNbeyoaqC2uYP2rr7XMMxCV5fLTk0gmJpIMDW0JpGVEroPpiZ4t0SyT8xLTSAjKX5Ej7FQAIiI9CEQNlYBfQ9o99ba0X0yKE6GRHPovq61k+MtndR5obLnaBN1zZ19DnaH1xBMCYXDNz84jwtm5fTb9nQoAEREoiQlMUBBYuiAuEh1dvdQ39pJnRcOJ0KirqWTutbQ8/qWTrJSEqJerwJARMRHCYG4sDWN0TUxhrJFRGTIFAAiIjFKASAiEqMUACIiMUoBICISoxQAIiIxSgEgIhKjFAAiIjHKXH9nOxqDzOwocMDvOvqRC9T4XcQAVN/wqL7hUX3DN5waZzjn8npPHFcBMJaZ2XrnXKnfdfRH9Q2P6hse1Td8I1GjNgGJiMQoBYCISIxSAETPKr8LGITqGx7VNzyqb/iiXqPGAEREYpTWAEREYpQCQEQkRikAImBmRWb2nJltN7OtZna7N/0uM6s0s43e7ZqwPl8xs3Iz22lm7xuFGveb2WavjvXetElm9rSZ7fbus32s76yw5bTRzBrM7A4/l6GZPWBmR8xsS9i0IS8zM1vkLftyM7vbonQR137q+zcz22Fmm8zsl2YW9KYXm1lr2HJc6VN9Q/59jnJ9j4bVtt/MNnrT/Vh+/X2ujN7foHNOt0FuwFRgofc4A9gFnAPcBXyxj/bnAG8BScBMYA8QGOEa9wO5vaZ9G7jTe3wn8K9+1derrgBwGJjh5zIELgUWAluGs8yA14ELAQN+C1w9gvW9F4j3Hv9rWH3F4e16/ZzRrG/Iv8/RrK/X/O8Cf+/j8uvvc2XU/ga1BhAB51yVc+4N73EjsB0oGKDLUuAR51y7c24fUA4sHvlK+6zjJ97jnwDXh033s74rgD3OuYGO6h7xGp1zLwLH+njdiJeZmU0FMp1zr7rQf+KDYX2iXp9z7vfOuRNXEX8NKBzoZ4x2fQMYE8vvBO8b8keBnw30M0a4vv4+V0btb1ABMERmVgycD6z1Jt3qrY4/ELaqVgAcDOtWwcCBEQ0O+L2ZbTCz5d60fOdcFYT+2IDJPtYX7gZO/ccbK8sQhr7MCrzHo10nwC2Evu2dMNPM3jSzF8zsEm+aH/UN5ffp1/K7BKh2zu0Om+bb8uv1uTJqf4MKgCEws3TgF8AdzrkG4D7gDGABUEVolRJCq2G9jfT+thc55xYCVwOfM7NLB2jrR32hFzZLBK4DHvMmjaVlOJD+6vGlTjP7KtAFPOxNqgKmO+fOB74A/NTMMn2ob6i/T79+zzdy6pcQ35ZfH58r/Tbtp5bTrlEBECEzSyD0S3rYOfcEgHOu2jnX7ZzrAe7nj5soKoCisO6FwKGRrM85d8i7PwL80qul2ls9PLEqe8Sv+sJcDbzhnKv26h0zy9Az1GVWwambYUa8TjP7BPB+4OPeKj/eZoFa7/EGQtuH54x2fafx+/Rj+cUDHwIeDavbl+XX1+cKo/g3qACIgLe98EfAdufc98KmTw1r9kHgxN4Gq4EbzCzJzGYCswkN0oxUfWlmlnHiMaGBwi1eHZ/wmn0C+LUf9fVyyjevsbIMwwxpmXmr6I1mdoH3d7IsrE/UmdlVwN8A1znnWsKm55lZwHs8y6tvrw/1Den3Odr1eZYAO5xzJzeb+LH8+vtcYTT/BqMxmj3Rb8DFhFapNgEbvds1wEPAZm/6amBqWJ+vEvoWsZMo7TUwQH2zCO0d8BawFfiqNz0HeBbY7d1P8qO+sNdMBWqBrLBpvi1DQkFUBXQS+hb1ydNZZkApoQ+6PcA9eEfYj1B95YS2A5/4O1zptf2w97t/C3gD+IBP9Q359zma9XnTfwys6NXWj+XX3+fKqP0N6lQQIiIxSpuARERilAJARCRGKQBERGKUAkBEJEYpAEREYpQCQEQkRikARERi1P8D2amdYgl52nEAAAAASUVORK5CYII=\n",
       "text/plain": [
        "STEP: 2000 MSE: 0.0023767943494021893\n"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
   ],
   "source": [
    "STEPS = 2000\n",
    "LEARNING_RATE = .02\n",
    "\n",
    "\n",
    "Xf = make_features(X)\n",
    "n_weights = Xf.shape[1]\n",
    "\n",
    "W = tf.Variable(np.zeros((n_weights, 1)), dtype=tf.float32)\n",
    "\n",
    "# For plotting\n",
    "steps, losses = [], []\n",
    "plt.figure()\n",
    "\n",
    "\n",
    "for step in range(1, STEPS + 1):\n",
    "\n",
    "    dW = compute_gradients(X, Y, W)\n",
    "    W.assign_sub(dW * LEARNING_RATE)\n",
    "\n",
    "    if step % 100 == 0:\n",
    "        loss = loss_mse(Xf, Y, W)\n",
    "        steps.append(step)\n",
    "        losses.append(loss)\n",
    "        plt.clf()\n",
    "        plt.plot(steps, losses)\n",
    "\n",
    "\n",
    "print(\"STEP: {} MSE: {}\".format(STEPS, loss_mse(Xf, Y, W)))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA+BUlEQVR4nO3dd3hUxfrA8e9seoeEUAOE3nsIXcBKkaaggHSUJtZ7vRfLvfZ21Z+ioohSBSnSBKQJ0ntCL6EHktASQklIT+b3x4lICckm2c1ml/fzPHlIduecfVkOb2bnzLyjtNYIIYSwfyZbByCEEMIyJKELIYSDkIQuhBAOQhK6EEI4CEnoQgjhIJxt9cKlSpXSwcHBtnp5IYSwS+Hh4XFa68CcnrNZQg8ODiYsLMxWLy+EEHZJKXXmXs/JkIsQQjgISehCCOEgJKELIYSDsNkYuhDCMaWnpxMdHU1KSoqtQ7Fr7u7uBAUF4eLiYvYxktCFEBYVHR2Nj48PwcHBKKVsHY5d0lpz+fJloqOjqVKlitnHyZCLEMKiUlJSCAgIkGReCEopAgIC8v0pRxK6EMLiJJkXXkHeQxlyERZzIzWDyMs3iIpP4kpSOgkp6SSlZeJsUrg4mfB0c6aMjxulfd0JDvCkhKerrUMWwqFIQhcFduJSIltOxBF+5grhZ64QczU5X8eX8XWjdllfQiqXpFW1ABoGlcDVWT40iqKzfv16XF1dad26dYHP4e3tTWJiogWjKjhJ6CJfTlxKYMHuGFYdusCp2BuAkZhDKvvTv0UlggO8qBzgib+XK74eLni6OJGRpcnIyiIxJYNLCalcuJbCqbhEIi4kcPjcdb744xj8AZ6uTnSoFUiXBuV4sHZpPF3l8hTWtX79ery9vQuV0IsT+R8j8pSRmcXvB84za/tZdkbG42RStKzqz5DWwXSsVZqgkh65jve5mhSumPB0daa0rzv1K/gBZW4+f+VGGjtOX2bT8ThWHbrI8gMX8HBxokuDcgxoWYnGFUvImKzIl549exIVFUVKSgovvfQSI0aMYOXKlbzxxhtkZmZSqlQpJk+ezMSJE3FycmLmzJl88803TJ48mccff5zevXsDf/e+ExMT6dGjB1euXCE9PZ0PPviAHj162PhveTdJ6OKe0jKyWLQnmgnrTnI2PonKAZ6M61yb3s2CKOXtZrHXKenlSqf65ehUvxzv9ajPztPxLNl3jiV7Y1iwO5p65X15tl0VujUsj7OTDMnYk3eXHuLwuesWPWfd8r683a1erm2mTJmCv78/ycnJNG/enB49evDcc8+xceNGqlSpQnx8PP7+/owaNQpvb2/++c9/AjB58uQcz+fu7s6iRYvw9fUlLi6Oli1b0r1792LX0ZCELu6itWb14Yt8+PsRzsYn0TDIj/88HsJDtUtjMln3AnYyKVpVC6BVtQDe7FqHRXti+HlbJK/M3cdXa44zpkM1ejUJkrF2kauvv/6aRYsWARAVFcWkSZN44IEHbs7p9vf3z9f5tNa88cYbbNy4EZPJRExMDBcvXqRs2bIWj70wJKGL25y4lMg7Sw6x+UQcNUp7M2VICB1rlbZJT8TbzZmBLSvzTGgl/jhykW//PMG/Fxzg23UnGNepDl0alC12PSRxu7x60tawfv161qxZw7Zt2/D09KRDhw40atSIo0eP5nmss7MzWVlZgJHE09LSAJg1axaxsbGEh4fj4uJCcHBwsVwJK90cAUBmluaHDSfp8vUm9kdf5Z1udVnxUjserF3G5knTZFI8Vq8sS8a2YeqQ5ni5OvP8L7vpPXEbu89esWlsovi5du0aJUuWxNPTk4iICLZv305qaiobNmzg9OnTAMTHxwPg4+NDQkLCzWODg4MJDw8H4LfffiM9Pf3mOUuXLo2Liwvr1q3jzJl7VrC1KaW1tskLh4SEaKmHXjycvZzEy3P3sPvsVR6tW4YPezUg0MdyY+SWlpml+TUsis9XHyMuMZV+oRUZ17kOfh7m17wQ1nPkyBHq1Kljs9dPTU2lZ8+exMTEUKtWLWJjY3nnnXdITk7mjTfeICsri9KlS/PHH39w7Ngxevfujclk4ptvvqFmzZr06NGDrKwsHnroIb755hsSExOJi4ujW7dupKen07hxY7Zs2cKKFSsIDg626rTFnN5LpVS41jokp/aS0O9zfxy+yKvz9qKA93vWp3uj8jbvkZsrMTWD8WuOMXnzaQK83Xi3ez0615dhGFuzdUJ3JPlN6GYNuSilOimljiqlTiilxuXSrrlSKlMp1TtfUYsil5GZxccrjvDcjDAqB3jy+4vt6NG4gl0lQ283Z97sWpffnm9LaR83xszazaiZ4VxOTLV1aELYRJ4JXSnlBEwAOgN1gX5Kqbr3aPcpsMrSQQrLSkzNYPj0MH7YcIpnWlRi/qjWVPT3tHVYBdYgyI/fnm/DuM61WRcRS6fxm9h4LNbWYQlR5MzpoYcCJ7TWp7TWacAcIKcZ9S8AC4BLFoxPWNiFayk8NXEbm0/E8VGvBnzYqwHuLk62DqvQnJ1MjGpfjcXPt6GkpwuDpuzk3aWHSEnPtHVoQhQZcxJ6BSDqlp+jsx+7SSlVAegFTMztREqpEUqpMKVUWGys9KCK2rGLCfT6bgtnLt9g8uAQ+reoZOuQLK5ueV+WjG3LkNbBTN0SSe+JW4mKT7J1WEIUCXMSek6DqnfeSf0K+LfWOtfukNZ6ktY6RGsdEhgYaGaIwhIOnbvG0z9sIzNLM29UKzrUKm3rkKzG3cWJd7rX46dBIZy5nES3bzez/qh8cBSOz5yEHg1UvOXnIODcHW1CgDlKqUigN/CdUqqnJQIUhbcv6ir9f9yBh4sT80a2ol55P1uHVCQerluGpWPbUtbXnaHTdvHVmmNkZdlmVpcQRcGchL4LqKGUqqKUcgX6AktubaC1rqK1DtZaBwPzgTFa68WWDlbk3+6zVxjw0w58PZyZO7IVwaW8bB1SkQou5cWiMW3o1aQCX605zsiZ4SSlZdg6LGFH1q9fz+OPPw7AkiVL+OSTT+7Z9urVq3z33Xf5fo133nmHzz//vMAx/iXPhK61zgDGYsxeOQLM01ofUkqNUkqNKnQEwmqOnL/OkCk78fd2Ze6IVnY9k6UwPFyd+KJPI97pVpe1Ry7SZ+I2Llwrfsu2RdHKzMz/DfPu3bszbtw9Z24XOKFbilnz0LXWy7XWNbXW1bTWH2Y/NlFrfddNUK31EK31fEsHKvInMu4GAyfvxNPVmZnDW1C+hIetQ7IppRRD2lRh8uDmnLmcRI8JmzkYc83WYQkriYyMpHbt2gwePJiGDRvSu3dvkpKSCA4O5r333qNt27b8+uuvrF69mlatWtG0aVP69Olzc8XnypUrqV27Nm3btmXhwoU3zztt2jTGjh0LwMWLF+nVqxeNGjWiUaNGbN26lXHjxnHy5EkaN27Ma6+9BsBnn31G8+bNadiwIW+//fbNc3344YfUqlWLhx9+2Kw6M+aQ4lwO6MK1FAZM3kFmVhZz7uOeeU461i7N/NGtGD4tjD4TtzG+b2MerVe8KuY5lBXj4MIBy56zbAPofO9hj78cPXqUyZMn06ZNG4YNG3az5+zu7s7mzZuJi4vjiSeeYM2aNXh5efHpp5/yf//3f/zrX//iueee488//6R69eo8/fTTOZ7/xRdfpH379ixatIjMzEwSExP55JNPOHjwIHv37gVg9erVHD9+nJ07d6K1pnv37mzcuBEvLy/mzJnDnj17yMjIoGnTpjRr1qzQb40U53IwN1IzGDZtF1dupDF9WCjVS/vYOqRip3ZZXxY/34ZaZX0YNTOcOTvP2jokYQUVK1akTZs2AAwYMIDNmzcD3EzQ27dv5/Dhw7Rp04bGjRszffp0zpw5Q0REBFWqVKFGjRoopRgwYECO5//zzz8ZPXo0AE5OTvj53T3ZYPXq1axevZomTZrQtGlTIiIiOH78OJs2baJXr154enri6+tL9+7dLfJ3lh66A8nM0rw0Zw8RF64zZUhzGgaVsHVIxVagjxu/PNeCMbN2M27hAS7fSGNMh2p2VfrALpjRk7aWO/8t//rZy8uYGKC15pFHHmH27Nm3tdu7d6/FrgOtNa+//jojR4687fGvvvrKKtea9NAdyMfLj7DmyCXe6V7PoeeZW4qnqzM/DgqhV5MKfLbqKO8uPSzTGh3I2bNn2bZtGwCzZ8+mbdu2tz3fsmVLtmzZwokTJwBISkri2LFj1K5dm9OnT3Py5Mmbx+bkoYce4vvvvweMG6zXr1+/qxzvY489xpQpU26OzcfExHDp0iUeeOABFi1aRHJyMgkJCSxdutQif2dJ6A5i1o4z/LT5NENaBzOoVbCtw7EbLk4mvujTiOFtqzBtayQvz91LWkaWrcMSFlCnTh2mT59Ow4YNiY+Pvzk88pfAwECmTZtGv379aNiwIS1btiQiIgJ3d3cmTZpE165dadu2LZUrV87x/OPHj2fdunU0aNCAZs2acejQIQICAmjTpg3169fntdde49FHH6V///60atWKBg0a0Lt3bxISEmjatClPP/00jRs35sknn6Rdu3YW+TtL+VwHEH4mnqd/2E7bGqWYPLg5TlbeJs5iMjMgOR6SLkN6MmSkQkb2n1mZ4OQCJicwuRjfu/mCRwlwLwGuXmDBj6xaayZuOMWnKyN4uE4ZJjzTBDdn+69xYwvFoXxuZGQkjz/+OAcPHrRpHIWV3/K5MoZu52ITUhkzazcVSnowvm+T4pXM05Mh7hjEn4IrkRB/Gq6choQLcCMOkq9wdxUJM5lcwDMA/IKMrxIVwa8iBFSDwNrgWyFfCV8pxegO1fB2c+I/vx1i5M/hTBzQzCEKl4n7hyR0O5aRmcULs3dzLTmdqUNCbbtjT9oNiA6D83uNaWoXDhjJXN8yfOFZCvyrGAk3OBC8AsGrFHiUNHrczu7ZX25Gzzwrw+jFZ6VDZhqkJhi/BJKvGn/eiIVr0cZrHV0BmbfUQXfzhcBaULouVGgKQc2N1zXlnqAHtgrGxcnE64sO8Oz0MH4cFIKHqyR1exMcHGz3vfOCkIRuxz5bfZTtp+L5ok8j6pb3LdoXT02AM1vhzBbjz3N7jAQM4BtkzBWu0x3K1DN6zSUqg7sVY9QaEi/B5eNw6QjERsClCDj8G+yebrRx9YbyTaBSK6jawUjyzq53napvaCVcnEy8Nn8fQ6buZMqQ5ni5yX+V/NBay4yhQirIcLhcpXbqz4iLNzeoeLJZUNG86OWTcHw1HFsJkVuMnrPJxegBt34BKrWGCs3AK6Bo4rmVUuBTxvgKvmU2g9bGkE90GETvgpgw2PQ5bPwfuHhC5dZGcq/VxfjFk+3JZkG4OJt4Ze5eBk3ZyfRhoXhLUjeLu7s7ly9fJiAgQJJ6AWmtuXz5Mu7u7vk6Tm6K2qFLCSl0/moTpX3dWfx8a+vevIs7Dgfmw6GFxhAKQKlaUPNRqP4IVAwFFzsrK5B81fhkcWq98fXX3yuwNtTuanyVawImEysOnOeF2XtoWqkk04Y1x9NVknpe0tPTiY6OJiVF6uUUhru7O0FBQbi43D6UKptEO5CsLM3QabvYfuoyy15oS40yVlgJmnAR9s+Fg/Ph/D5AGb3eOt2gxqPGOLgjuXrWGIOPWGZ88tCZxg3WBn2gUV+Wnffhxdl7aFk1gClDmsuNUmFTktAdyJTNp3lv2WHe71mfgS1znh9bIFlZcHo9hE2Fo8uN8fDyTY2kVq8X+Jaz3GsVZ0nxcGyV8cvs5J/GTd1yjdkf0Imh4cHUr1GNSYOayZRGYTOS0B3EkfPX6fHtFh6oWYofB4VYZnwy5bpx03DXT8bUQs8AaNwfmg6GUjUKf357lnDRSOz75sCF/WSaXFiSHkpE0NP8Y9gAXKWnLmxAEroDSM3IpPs3W4hPSmPlS+0I8HYr3AmvxcCO7yF8OqReh8ptIGSYMaziXMhzO6JLRyB8GmnhM3HNSCTatSrlHh6LU+O+xpRLIYqIJHQH8Pmqo3y77gRThzSnY+1C1Gm5fBI2fg4H5hkzQOr1hFZjjZkqIm+piWxaNBH/wzOoZzqD9iiJCh0BoSOMOfVCWJkkdDt3MOYaPSZsoWfjCnzxVKOCneRKJGz8DPbOBidXaDYEWo6GkhYch7+PfLv2GOvWLOPD0n9S+9omcPaAJgOg9VgoGWzr8IQDk6X/diwtI4vX5u/H38uV/z5eN/8nSLgI6z+GPT+DcjJ6km1fMeZriwJ7/sEaXEnuRKfNtXiv9UsM0oshfBqETYaGfaH9vxxvNpAo9iShF3Pfrz/JkfPXmTSwGX6e+Vjan54M276FTV8ay+abDYF2/wDf8laL9X6ilOLNLnW4mpTOf7dGQ4/XGPTym7D1WyOpH5hn9NgfeM2oNSNEEZCEXoxFXLjOt+uO071RefO3SdMaDi6ANe/AtSio/Tg88t5tqyCFZZhMik+fbMC15HTeXnIIP4/G9Oj0kbFqdtMXRo997y/QbKiR2L0DbR2ycHBSD72YysrSvL7wAL7uLrzTvZ55B8UehWldYcFwo+DV4GXQd5YkcytydjLxbf8mNA/25x/z9rHu6CVjzn7Xz+HF3dCorzEl9JumsPkrozSwEFYiCb2YmrMrij1nr/Jm1zr4e91dQOo26cnw5wfwfRu4eAi6jYcR66GKZYrmi9y5uzjx0+AQapX1YfTMcMIi440nSlSC7t/AmO1GzZg1b8O3zeHQYuOTlBAWJgm9GIpLTOWTFUdoWdWfXk0q5N741Ab4vrUxg6X+EzA2zBgvz6NMrLAsX3cXpg8LpZyfB8Om7eLYxb+3ISOwJvSfCwMXg5sP/DoYpnY2yv4KYUGS0Iuhj5YfITk9kw961r/3atC0JFj+GszI3i184GJ4YpKM09pQKW83ZgwLxc3FicFTdnLh2h3Fqap1hJEbodvXRtGzH9rDyjeMUsRCWIAk9GJm+6nLLNwdw4gHqlK99D0Kb0XthIltYeckaDEaRm0xkoWwuYr+nkwb2pzryekMmbqT6ynptzcwOUGzwTB2FzQdBNsnwLehRt12GYYRhSQJvRhJy8jircUHCSrpwdiOOdRRyUyHte/BlMeMqYiDl0LnT8DVs+iDFfdUr7wfEwc248SlREbOCCc1I/PuRp7+0O0rGL7GqJ8zbxD88pSxA5MQBSQJvRiZsS2SE5cSebd7vbu3PbsaBVO7GNPhGvWH0VuhygO2CVTkqV2NQP7XuyHbTl3mtV/3k5V1j953xebGDezHPoLIzfBdK9g9Q3rrokAkoRcTlxNTGb/2OO1rBvJQnTtWcUYsN4ZYLh2BJydDzwnW3c5NWMQTTYN47bFaLNl3jk9XRty7oZMztHre+CVdtiEseQFmPim9dZFvktCLiS/+OEZSWib/ebzO3w9mpBk3zeb0M6bAjdwADXrbLkiRb2M6VGNgy8r8sPEU07aczr2xfxVjGK3L53B2O0xoaVTDlN66MJMk9GLgyPnrzNl5lkGtKv99IzQxFn7uadw0a/4cDP9DFgjZIaUU73Svx6N1y/DussOsOHA+9wNMJgh9DkZvgfKNYemLMKc/3LhcJPEK+2ZWQldKdVJKHVVKnVBKjcvh+R5Kqf1Kqb1KqTClVNucziPuprXmvaWH8fNw4eWHahoPntsLkzpATDg88aOx6tAlf5vFiuLDyaT4ul8TmlQswUtz97LzdHzeB/lXgUFLjLH1E2uMtQan1ls9VmHf8kzoSiknYALQGagL9FNK3Vn2by3QSGvdGBgG/GThOB3WqkMX2XbqMq8+UtMovnVgvjGLBQ3DVkLDp2wdorAAdxcnJg9uTlAJD56bEcbJ2MS8DzKZjLH1Z9ca90xm9IQ//msMxQmRA3N66KHACa31Ka11GjAH6HFrA611ov67sLoXIIN+ZkjNyOTD5YepVcaHfs0rwtr3jTos5ZsYMx/KN7F1iMKCSnq5Mn1YKC5OiqFTd3E50cy6LuUawogNxgrgLeNh8iPGRiVC3MGchF4BiLrl5+jsx26jlOqllIoAfsfopd9FKTUie0gmLDY2tiDxOpSft50hKj6ZtzpVw3nJaNj0OTQZaHzU9i7ErkSi2Kro78mPg0K4eD2FZ2eEkZKewxz1nLh6GvPWn55pbFYyqQMcWWbFSIU9Mieh57T2/K4euNZ6kda6NtATeD+nE2mtJ2mtQ7TWIYGB9/cS9esp6UxYd4LHqrnTbsdI2D8XHnzLKObknEcxLmHXmlQqyfi+jdkbdZVX5+299xz1nNTpZpQP8K8Kc58xhmAyM6wXrLAr5iT0aKDiLT8HAefu1VhrvRGoppSSDRZzMWnDKdyTLjA+aRyc3Qa9fjBqZt+rdotwKJ3ql+ONznVYfuBC7nPUc1KyMgxbZdRZ3zIeZvQwdqYS9z1zEvouoIZSqopSyhXoCyy5tYFSqrrKriKllGoKuAIyz+oeLl1PYc3mzSz3fg/3pAswYIFRN1vcV55tV+XmHPVZO87k72AXd2MIpudEYzbUD+3gzDarxCnsR54JXWudAYwFVgFHgHla60NKqVFKqVHZzZ4EDiql9mLMiHla22r3aTswZ9kKZprewddFw9DlULWDrUMSNqCU4u1udelYK5D//naI9Ucv5f8kjfvBc2vB1QumdzMWIon7lrJV3g0JCdFhYWE2eW1bijmwEe/5fcHVC7+Ry6FUDkW4xH3lRmoGfSZu48zlG/w6qjV1yxegrEPyFfh1KJxaBy1GwaMfGiUFhMNRSoVrrUNyek5Wihal05sIWPgU1/Amc4gkc2HwcnNmypDm+Hq4MGzaLs5fS87/STxKwjPzoeUY2DERZvU2kry4r0hCLyqnNpA180nOZAawssU0/CtIMhd/K+vnzpQhzUlMzWDYtDASUwswc8XJGTp9DN2/NSo3/vggxB6zfLCi2JKEXhQit8DsvsSYyjHa+V36PxRq64hEMVSnnC8TnmnKsYsJjP1lNxmZWQU7UdOBRpGvlOsw+WEjuYv7giR0azu7A2b1IdmzHL0S/k3fDk3wdpOxTZGz9jUDeb9HfdYfjeXtJYco8D2uyq3guT/Buwz83MsoKSEcniR0a4oJN8YyfcryT8/3wTuQAS0r2zoqUcz1b1GJUe2rMWvHWX7cdKrgJ/prvnqFEKOkxOavpBSvg5OEbi0XDho9I09/9nScwe+nYVT7ani6Su9c5O1fj9Wia8NyfLQ8Iu+Su7nx9IeBi6DeE7Dmbfj9H7Ky1IFJdrGGK5Ew8wlw9YZBS/j013ME+rhJ71yYzWRSfNGnEeevJvPy3L2U8XOnaaWSBTuZi7ux01WJisbK0uvnoPcU2YvWAUkP3dJuxMHPT0BGKgxYwNZ4L7afimdMh2q4uzjlfbwQ2dxdnPhxUAhl/dx5bnoYZy8nFfxkJhM88p6xG9Kxlcanx+SrFotVFA+S0C0pNRFm9YHrMdB/HjqwNl/+cYwyvm70C61k6+iEHQrwdmPqkOZkas2QaTu5mlTIWuihz0Gfqcb9nemPQ2IBVqeKYksSuqVkpsO8QXB+H/SZBpVasOXEZXZFXuH5jtWldy4KrGqgNz8OCiH6SjIjZoSbX3L3Xur1gv5zIO4ETOkEV6PyPkbYBUnolqA1/P4qnFxrFEyq1RmtNV+tOUY5P3eebl4xz1MIkZvmwf580acROyPjeW3+/vyV3M1J9Ydh0GJjiHDKY7IAyUFIQreEbRNg9wxo909oOgiAHafjCTtzhVHtq+HmLL1zUXjdGpVnXOfaLN13js9WHy38CSu1hKG/Q2YaTO1k7GUr7Jok9MI6ugJWvwV1e0DHN28+PGHdCUp5u0rvXFjUyAeq8kyLSny//iS/7Dhb+BOWbWDMVXfxgundITq88OcUNiMJvTAuHID5w6FcI6Mutcl4O/dFXWXT8TiGt60qY+fCopRSvNu9Hg/WLs1/fjvIuoKU3L1TQDWjjLNnSfi5J0TtKvw5hU1IQi+oxEvwS19w94N+c26b0/vtuhP4ujszoKXMbBGW5+xk4pt+TahTzofnZ+3mYMy1wp+0REUY8jt4BhhTGs/uKPw5RZGThF4QmRkwfxgkxUG/2eBb7uZTEReu88fhiwxpUwUfdxcbBikcmZebM1MGN6ekpytDp+0i5moBSu7eyS/I6Kl7lzYWxskOSHZHEnpBrH0XIjfB419B+ca3PfX9+pN4ujoxtHWwLSIT95HSvu5MHdqclPRMhk7dybXk9MKf1Le80VP3KQcznzQqhQq7IQk9vw7/Blu/hpDhxvZft4iMu8HSfecY0LIyJb1cbRSguJ/ULOPDDwOacTruBqNnhpOWUcCSu7fyLQdDloFfBaO4nPTU7YYk9PyIPQaLx0BQc+j0yV1PT9xwEmcnE8+2rWKD4MT9qnX1UnzyREO2nrzMuIX7C15y91Y+ZY2eul+Qsfo5Rma/2ANJ6OZKTYS5A8DZHfpMB+fbe+DnryWzYHc0T4dUpLSvu42CFPerJ5sF8crDNVm4O4av1hy3zEm9S8Og38ArwKhPdOGgZc4rrEYSurlW/AvijhlV6vwq3PX0lM2nydIw4oGqNghOCHjxoer0bhbE+LXH+TXMQsv5fcvDoCXg6gUzesiK0mJOEro5DsyHvbPggdegavu7nr6eks7snVF0aVCOiv5SklTYhlKKj59oQNvqpXh94QE2H4+zzIlLVjaSujLBjO4QX4hNN4RVSULPS/xpWPYKVGwB7f+dY5PZO86SmJrBSOmdCxtzcTLx3YCmVC/tzeiZ4Rw5f90yJy5V3Rh+yUiF6T3gWrRlzissShJ6bjLTYcGzgIInfzJ2Vb9DWkYWU7dE0rpaAPUr+BV9jELcwdfdhSlDmuPl5szgKTuJii9EHfVblakLAxdCylVj+OXGZcucV1iMJPTcrP8YYsKg+3gokfOqz6X7znHhegrPSe9cFCPlS3gwY3goKemZDJ6yk/gbhayjfvPETaD/PKOHPqu3MVlAFBuS0O8laids/hKaDDDqR+dAa82Pm05Rq4wPHWoGFnGAQuSuZhkfJg9pTszVZIZO20VSmoX2Eq3cCnpPNWr/zxsIGRb6ZSEKTRJ6TtKTYfFo8K0Aj318z2Ybj8cRcSGBZ9tVQSlVhAEKYZ7mwf58068JB6KvMmbWbtIzLbDwCKB2F+g2Hk7+afxfybLQeUWhSELPyZ8fwOUT0P0bcPe9Z7MfN56ijK8bPRrfPY1RiOLi0Xpl+bBXA9YfjeXfCyy08Aig6UB46G04OB9WvWFs9CJs6u67fPe7M9uMDStChkO1jvdsdjDmGptPxPHvTrVxdZbfi6J46xdaiUvXU/lyzTECfdx4vXMdy5y47StG5dEd3xsLkdq9apnzigKRhH6rtBvw2xijlOgj7+Xa9KdNp/BydaJ/CymRK+zDiw9V51JCCj9sOEVpH3eGW6JEhVLw2EdG5dG174J3GWjyTOHPKwpEEvqt1n1kLJoYvAzcvO/ZLOZqMkv3n2dI62D8PKRErrAPSine61Gfy4lpvL/sMKW8XS0zXGgyQY/vjP1Jl75orKSu2qHw5xX5ZtZYgVKqk1LqqFLqhFJqXA7PP6OU2p/9tVUp1cjyoVrZ+X2w/TtoNhSqtMu16fStkQAMkyJcws44mRRf9W1MaBV//vnrPjYdj7XMiZ1d4anpUKomzB0ElyIsc16RL3kmdKWUEzAB6AzUBfoppere0ew00F5r3RB4H5hk6UCtKivTWA3qGQAPv51r0xupGczeeZZO9ctSoYRHEQUohOW4uzjx46AQqgV6M/LncMLPXLHQif2MOeou7kaFxoSLljmvMJs5PfRQ4ITW+pTWOg2YA/S4tYHWeqvW+q+rYjsQZNkwrSxsilEe9LGPwKNkrk0X7o4mISWDYW2CiyY2IazAz8OFGcNCCfRxY+jUnRw+Z6ESASUqQv+5xpj67L6QZqFVqsIs5iT0CsCtpduisx+7l+HAipyeUEqNUEqFKaXCYmMt9FGvsBIuwNr3jDG/Bn1ybZqVpZm6NZKGQX40rZR74heiuCvt687M4S3wcnNm0JQdnIq10KrP8k3gyclwbg8sfM74BCyKhDkJPacVMzlOOFVKdcRI6DlWsdJaT9Jah2itQwIDi8nKylVvGAWHuv6fccc+FxuPx3Iq9gZD2wTLQiLhECr6e/Lz8BZoDQN+2mGZvUnBWHjU6ROIWAar/2OZc4o8mZPQo4GKt/wcBJy7s5FSqiHwE9BDa20fVXvObIWDC6DtyxBQLc/mU7dEEujjRtcG5a0fmxBFpHppb6YPCyUhNYOBP+0gNiHVMiduOQpajILtE2DXZMucU+TKnIS+C6ihlKqilHIF+gJLbm2glKoELAQGaq3towJ+VhasHGcs72/zcp7NT1xKZMOxWAa0qCwLiYTDqV/Bj6lDmnP+WgqDpuzkWpIFNpwG475UjUeNDWJOb7LMOcU95ZmZtNYZwFhgFXAEmKe1PqSUGqWUGpXd7L9AAPCdUmqvUirMahFbyr5fjKmKD78LrnlvSjF9aySuTiZZSCQcVkiwPz8MbMaJSwkMnbaTG6kWKOZlcjJKT/tXhXmDjP0FhNUoi9V1yKeQkBAdFmajvJ+aAN80gxKVYfjqPMfOryWn0+rjtXSuX44vnrK/KfZC5MeKA+d5/pfdtK5Wip8Gh+Du4lT4k14+CT8+aGxpN3w1uPkU/pz3KaVUuNY6JKfn7s+xg01fQOJF46aNGTc35+2KIiktk6EyVVHcBzo3KMf/ejdiy8k4Rv4cTmqGBWapBFSDPtMg9igsHCnVGa3k/kvo12Jg23fQ8GkIapZn88wszfRtkYRW8ZcdicR9o3ezID7u1YANx2IZPXO3ZZJ6tY7Q6WM4+jus+6Dw5xN3uf8S+oZPAA0d3zSr+R+HLxJ9JVkWEon7Tt/QSnzYqz5/Rlxi7C97SMuwQK86dAQ0HWR8Sj4wv/DnE7e5vxJ63AnYMwtChhk7mZth2tbTVCjhwSN1y1o5OCGKn2daVOa9HvX44/BFXpy9p/AbZCgFXb6ASq3gt+eNiQnCYu6vhL7uA3B2h3b/NKv50QsJbD8Vz6BWlXEyyUIicX8a1CqYt7vVZeWhC7w8Zy8ZhU3qzq7w1M/g4Q9zB0JSvGUCFfdRQj+3Fw4tglZjwNu8Vaozt5/B1dnEUyEV824shAMb2qYKb3Wtw+8HzvPKvH2FT+regfDUDLh+Lrs8gNwktYT7J6Gvyy681foFs5onpKSzcHc03RqWp6SXq5WDE6L4e7ZdVV7vXJul+87xqiWSesXm0PlTOLEm+96WKKz7Y4OL8/vg+Cro+JZR4tMMi/fEcCMtk4GtzBtrF+J+MLJ9NbI0fLoygrSMLL7u16RwK6dDhhmVTjd8CuWbQq1Olgv2PnR/9NA3fQFuvhD6nFnNtdb8vP0MDYP8aFyxhHVjE8LOjO5Qjf8+boypj5oZTkp6IaY0KgVdv4ByjWDhCGMBkigwx0/osUfh8BIjmXuUMOuQHafjOXYxkQEtpXcuRE6Gta1yc0rjs9PDSEorRJkAFw/jJqnJZNwklRrqBeb4CX3zl8YF03KM2Yf8vP0Mfh4udG8kVRWFuJdnWlTm8z6N2HoyjiFTdpGQUoiCXiUrGzVfLh2GpS+BjUqS2DvHTuhXImH/PGOfUK9SZh1y6XoKqw5e4KmQIMvUsBDCgfVuFsT4vk0IP3uFgZMLWaWx+sPGgr8D84xdxES+OXZC3/49KBO0Hmv2IbN3RpGRpXmmhQy3CGGObo3K8/0zTTl87jr9ftxOXGIh6qm3+4eR2Fe+Duf3Wy7I+4TjJvSU68aq0Hq9jApvZkjPzOKXnWdoXzOQ4FJeVg5QCMfxaL2yTBrUjFNxifT+fitR8QUcBzeZoNcP4OkPvw4x/h8LszluQt87C9ISjF1TzLTm8EUuXk9loNwMFSLfOtQqzaxnW3IlKZ0nvt/KkfMFTMZepaD3FLhyWsbT88kxE3pWJuz4AYJCoULeFRX/8vP2M1Qo4UHH2qWtGJwQjqtZ5ZL8OqoVTkrx1A/b2Hm6gMv6K7eGB9+CQwshfKplg3RgjpnQj682frvno3d+4lICW09e5pmWlaRuixCFULOMDwvGtCbQx42Bk3fwx+GLBTtRm1eg2kOwYhxcOGDZIB2UYyb0HRPBpzzU6W72ITO3n8XVycTTUrdFiEKrUMKD+aNaU7usD6NmhjMvLCr/J7l1PH3eYGOnMZErx0vo8afh1HpoNgScXMw6JCktgwXh0XRpUJYAbzerhifE/cLfy5VfnmtJ62oB/Gv+fsavOU6+t7z0DoQnJ2ePp78s4+l5cLyEvncWoKBxf7MPWbrvHAmpGbIyVAgL83JzZvLg5jzRtAJfrjnGP37dl/+NMoLbQMc34OB82POzdQJ1EI6V0LMyYe8vUO1BKGH+0MkvO6OoWcabZpVLWjE4Ie5Prs4mvujTiFcersnC3TEMmrIj/wuQ2v4DqjwAK/5tbFQjcuRYCf3kn3A9BpoONPuQQ+eusS/qKv1CK6HM2DBaCJF/SileergGXz7diN1nrtLr+y2cvZyPuep/jac7u8GC4ZCRZr1g7ZhjJfTdM4xdUGp1MfuQ2TvP4uZs4okmQVYMTAgB0KtJED8PDyX+Rhq9vtvC7rNXzD/Ytzx0/wbO74V1H1otRnvmOAk9+QocXQENnzZ+i5vhRmoGi/eco2vDcvh5mncDVQhROC2qBrBwdGu83Z3pO2k7C8KjzT+4TjdjwsOW8XB6o9VitFeOk9AjlkNWOjTobfYhy/afIzE1g2daVLJiYEKIO1UN9GbRmDY0q1SSf/y6j/eWHjZ/B6THPoKA6rBwpOxHegfHSeiHFkKJSvlaGfrLjrPULONN00pyM1SIoubv5cqM4aEMaR3MlC2nGTx1J1dumDE27uoFvSfDjVhY8oJMZbyFYyT0pHhj7nm9XsYOKGY4GHONfdHX6C83Q4WwGRcnE+90r8f/ejdk1+krdJ+wmYgLZtSAKdcIHvovRCyD3dOtH6idcIyEfmQpZGVAvSfMPuSvm6G95GaoEDb3VEhF5o5sSVpGFk98t5XlB87nfVCrsVC1g1FqN+641WO0B46R0A8vBv+qxm9tM9xIzeC3ved4vGF5uRkqRDHRpFJJlo5tS+2yPoyZtZt3lx7KfRGSyQQ9JxqTIBaNhMxCbIPnIOw/oacmQuRmqN3V7OGWpfuMm6H9W0jdFiGKk9K+7swZ0YphbaowdUskfX7YRvSVXOar+5aDx7+EmHBju8n7nFkJXSnVSSl1VCl1Qik1LofnayultimlUpVS/7R8mLk4vREy06D6I2YfMnvnWWqV8ZGboUIUQ67OJv7brS7fP9OUU5cS6fr1ZtYeyaViY71eUL83bPgEzu0tsjiLozwTulLKCZgAdAbqAv2UUnXvaBYPvAh8bvEI83J8Nbh6Q6VWZjX/62Zov9CKcjNUiGKsc4NyLHuxLUElPRg+PYxPVkSQfq+pjV0+A69AWDQK0lOKNtBixJweeihwQmt9SmudBswBetzaQGt9SWu9CyjEDrEFoDUc/8O4MeLsatYhN2+GNpWboUIUd5UDvFgwujX9W1Ri4oaT9J64jci4G3c39PSH7t9C7BFY90HRB1pMmJPQKwC3FjOOzn7M9i4dgevRUONRs5rfdjPUQ26GCmEP3F2c+KhXAyb0b0pk3A26fL2JubvO3l2Kt8bDEDIMtn4LkVtsE6yNmZPQcxqXKNBMfqXUCKVUmFIqLDY2tiCnuF3kJuPPah3Nav73zVBZGSqEvenasBwrX25Ho6AS/HvBAUbP3H33QqRH3oeSlWHx6PtyQwxzEno0cOt0kCDgXEFeTGs9SWsdorUOCQwMLMgpbndmK/gGGStEzfDLzZuhJQr/2kKIIlfOz4NZz7bg9c61WRtxkU7jN7Lx2C2dQzdvoyrj1bOw6k3bBWoj5iT0XUANpVQVpZQr0BdYYt2wzKA1nN0Olc2/Gbo/+hr9W8jKUCHsmcmkGNm+GovGtMHH3YVBU3YybsF+rqdk38Kr1BLavGisID222rbBFrE8E7rWOgMYC6wCjgDztNaHlFKjlFKjAJRSZZVS0cCrwFtKqWillK81A+fKaUi8YPzjmWHurijcnE30bFw8hv+FEIVTv4Ify15oy8j2VZkXFsWj/7eRdRGXjCc7vgmBdWDpS5ByzbaBFiGz5qFrrZdrrWtqratprT/Mfmyi1npi9vcXtNZBWmtfrXWJ7O/NKMhQCGe3G39Wap1n05T0TBbvjaFLAymTK4QjcXdx4vXOdVg0pg2+Hs4MnbaLV+ft5Wqagp4TjE7f6rdsHWaRsd+VojHh4OYLgbXzbLri4HkSUjJ4KkRWhgrhiBpVLMHSF9rywoPV+W3vOR75ciNL48qhW71gbHxz8k9bh1gk7DehXzgIZeoZ9RzyMHdXFJUDPGlZ1b8IAhNC2IKbsxP/eLQWvz3fhjK+brwwew/DzzxMWomqsOQlo0yIg7PPhK41XDxkJPQ8nLl8g+2n4nkqRFaGCnE/qF/Bj9+eb8u73euxKzqZQXGD0deiyFj9tq1Dszr7TOhXz0BaApSpn2fTeWFRmBQ8KStDhbhvOJkUg1sHs/af7SlTvz1TMx7DOfwndm9cdveCJAdinwn9wkHjzzwSekZmFr+GRdOhVmnK+rkXQWBCiOKktI874/s2oc6AzzinyuK/5lWe/WmjeZto2CH7TOiXs4vZl879huiGY7FcSkjl6eZyM1SI+1mr2pUIfGYSwaaLtI+ZRJfxm3h94QFiE1JtHZpF2WdCv3oWPPzBzSfXZnN3RVHK240Ha5cuosCEEMWVS/X2EDKMgfzOGw0T+TUsio6fr2fCuhMkp2XaOjyLsNOEHgUlcu91xyak8mfEJZ5sWgEXJ/v8awohLOyR91B+QTx7+QtWv9iCllUD+GzVUR74bB3Tt0aSmmHfid0+M93Vs3nWb1m4O5qMLE0fmXsuhPiLm4+xw1HcUapG/MRPg0OYN7IVVUp58faSQzz4+Qbm7Yoi415114s5+0voWsO1KPC7d0LXWjN3VxQhlUtSvbR3EQYnhCj2ajwC9Z+ETZ9D7DFCq/gzd0RLZgwLpZS3K/9asJ9HvtzIoj3RdpfY7S+hp92A9CTwKXPPJmFnrnAq7gZPyc1QIUROOn0CLh6w7GXIykIpxQM1A1n8fBsmDWyGm7OJV+buo+MX65m5/Qwp6fYxFGOfCR2MbefuYe6uKLzdnOnaoFwRBSWEsCvepY3a6We2wN6ZNx9WSvFovbIsf7EdkwY2w9/LjbcWH6Td/9bxw4aTJKZm2DDovNlhQs9evnuPhJ6Qks7v+8/TrVE5vNycizAwIYRdaTLQKO63+j+QeOm2p0wmI7EvHtOaX55tQa0yPny8IoLWH6/lkxURxFxNtlHQubPDhP5XD90rx6eX7T9PcnqmFOISQuTOZIJuXxlDuCtfz7GJUorW1Usx89kW/PZ8G9pUL8WkjSd54H/rGDMrnJ2n44vVylP768LmkdDn7oqiZhlvGlcsUXQxCSHsU2AtaPsqbPgEGvUz9iW9h0YVS/D9gGZEX0ni5+1nmLMziuUHLlC3nC9D2gTTrWF5PFydijD4u9lxD/3uIZejFxLYG3VVCnEJIczX7lUIqAG/v/J3fslFUElPXu9ch+2vP8THTzQgM0vzr/n7Cf1wDW8tPsCB6Gs267XbYULP3vg1hx763F1RuDgpnpBCXEIIczm7QbfxxvqW9R+bfZiHqxP9Qiux8uV2zB3RkkfqluHXsGi6fbuZrl9vZvrWSK4lpVsx8LspW/0mCQkJ0WFhYfk/8OpZOLMNanUG9793uUvNyKTlR2tpXa0UE55pasFIhRD3hSUvwJ5ZMGI9lGtYoFNcS05nyb5zzN11loMx13F1MtGxdiA9GlfgwdqlcXcp/JCMUipcax2S03P2N4ZeolKOq0TXHL7ElaR0mXsuhCiYR96DiOXw+z9g2CqzNs+5k5+HCwNbVmZgy8ocjLnGgt3RLNt/nlWHLuLt5sxj9crSo3F5WlcLwNkKJUnsb8jlHuaGRVHez5221UvZOhQhhD3yKGkk9eidt81NL6j6Ffx4u1s9tr/+ELOebUGXBmVZffgCg6bs5L1lhy0Q8N3sr4eeg+grSWw6HssLD9bAySQ3Q4UQBdSon7EH6R9vQ+3HwbPw21Y6mRRtqpeiTfVSvNejPuuPxlLR38MCwd7NIXro88OjAejTTG6GCiEKwWSCrl9AyjVY+67FT+/u4kSn+mWpV97P4ucGB0joWVmaX8OiaVOtFBX9PW0djhDC3pWtDy1GQfh0iA63dTT5YvcJfcvJOGKuJsuuREIIy+kwDrzLGHPTs+yjMBc4QEKfuyuKEp4uPFrv3tUXhRAiX9x94bEP4fw+CJti62jMZtcJ/cqNNFYfukjPxhVwc7btklshhIOp/yRUaQ9r37+reFdxZdcJfdGeGNIys2S4RQhheUoZN0jTk+CP/9o6GrPYbULXWjMvLIqGQX7UKeeb9wFCCJFfpWpA6xdg32yI3GLraPJktwl9f/Q1Ii4kSJlcIYR1PfAa+FWEFf+CTNngwirmhkXh7mKie+Pytg5FCOHIXD3h0Q/g4kHYPc3W0eTKLhN6UloGS/eeo0uDcvi6u9g6HCGEo6vbA4LbwZ8fQFK8raO5J7MSulKqk1LqqFLqhFJqXA7PK6XU19nP71dKWbXc4fIDF0hIzeBpGW4RQhQFpaDzp8YK0nUf2Tqae8ozoSulnIAJQGegLtBPKVX3jmadgRrZXyOA7y0c523m7YqiSikvQqsUvs6CEEKYpUw9aP4shE2GCwdtHU2OzOmhhwIntNantNZpwBygxx1tegAztGE7UEIpVc7CsQJwKjaRnZHx9AkJkl2JhBBFq8Pr4F4CVo6DYrSX6F/MSegVgKhbfo7Ofiy/bVBKjVBKhSmlwmJjY/MbKwAnLiXi7+VKb9mVSAhR1Dz94cE3IXITHP7N1tHcxZyEnlM3+M5fTea0QWs9SWsdorUOCQwMNCe+uzxaryw733iI0r7uBTpeCCEKpdlQKNMAVr8FaUm2juY25iT0aODWu49BwLkCtLEYa+z0IYQQZjE5GTdIr0XB1q9tHc1tzMmMu4AaSqkqSilXoC+w5I42S4BB2bNdWgLXtNbnLRyrEEIUD8FtoN4TsPlLY5/jYiLPhK61zgDGAquAI8A8rfUhpdQopdSo7GbLgVPACeBHYIyV4hVCiOLh0fcBBav/Y+tIbjJrCzqt9XKMpH3rYxNv+V4Dz1s2NCGEKMb8gqDtK7D+Izi9Caq0s3VE9rlSVAghioU2Lxp1Xla9AVlZto5GEroQQhSYiwc89DZc2A/759g6GknoQghRKPWfhArNjI0w0m7YNBRJ6EIIURgmEzz2ESScg63f2jYUm766EEI4gkotjYqMW76C67absS0JXQghLOHhdyArA9Z9YLMQJKELIYQl+FeF0BGwZxac32+TECShCyGEpTzwGniUhNVv2qQaoyR0IYSwFI8SRond0xvh2Koif3lJ6EIIYUkhQyGghlGNMTO9SF9aEroQQliSk4tR5+XycQibWqQvLQldCCEsrWYnqPIArP8Ykq8W2ctKQhdCCEtTCh79AJLjYcv4IntZSehCCGEN5RpBg6dg+/dw3Wr7/dxGEroQQljLg2+CzoR1HxXJy0lCF0IIaykZDM2fhb2z4FKE1V9OEroQQlhTu3+CqzesecfqLyUJXQghrMkrANq+DMdWwJmtVn0pSehCCGFtLUaDT3n4479WLQkgCV0IIazN1RM6vg7Ru+DIUqu9jCR0IYQoCo36Q2BtWPuu1UoCSEIXQoii4ORs1Ey/fAJ2z7DKS0hCF0KIolKzE9TvbZTYtQJnq5xVCCHE3ZSC3pOtdnrpoQshhIOQhC6EEA5CEroQQjgISehCCOEgJKELIYSDkIQuhBAOQhK6EEI4CEnoQgjhIJS2YuWvXF9YqVjgTAEPLwXEWTAcSymucUHxjU3iyh+JK38cMa7KWuvAnJ6wWUIvDKVUmNY6xNZx3Km4xgXFNzaJK38krvy53+KSIRchhHAQktCFEMJB2GtCn2TrAO6huMYFxTc2iSt/JK78ua/isssxdCGEEHez1x66EEKIO0hCF0IIB1HsErpSqpNS6qhS6oRSalwOzyul1NfZz+9XSjU191grx/VMdjz7lVJblVKNbnkuUil1QCm1VykVVsRxdVBKXct+7b1Kqf+ae6yV43rtlpgOKqUylVL+2c9Z8/2aopS6pJQ6eI/nbXV95RWXra6vvOKy1fWVV1xFfn0ppSoqpdYppY4opQ4ppV7KoY11ry+tdbH5ApyAk0BVwBXYB9S9o00XYAWggJbADnOPtXJcrYGS2d93/iuu7J8jgVI2er86AMsKcqw147qjfTfgT2u/X9nnfgBoChy8x/NFfn2ZGVeRX19mxlXk15c5cdni+gLKAU2zv/cBjhV1/ipuPfRQ4ITW+pTWOg2YA/S4o00PYIY2bAdKKKXKmXms1eLSWm/VWl/J/nE7EGSh1y5UXFY61tLn7gfMttBr50prvRGIz6WJLa6vPOOy0fVlzvt1LzZ9v+5QJNeX1vq81np39vcJwBGgwh3NrHp9FbeEXgGIuuXnaO5+Q+7VxpxjrRnXrYZj/Bb+iwZWK6XClVIjLBRTfuJqpZTap5RaoZSql89jrRkXSilPoBOw4JaHrfV+mcMW11d+FdX1Za6ivr7MZqvrSykVDDQBdtzxlFWvr+K2SbTK4bE751Xeq405xxaU2edWSnXE+A/X9paH22itzymlSgN/KKUisnsYRRHXbozaD4lKqS7AYqCGmcdaM66/dAO2aK1v7W1Z6/0yhy2uL7MV8fVlDltcX/lR5NeXUsob4xfIy1rr63c+ncMhFru+ilsPPRqoeMvPQcA5M9uYc6w140Ip1RD4Ceihtb781+Na63PZf14CFmF8vCqSuLTW17XWidnfLwdclFKlzDnWmnHdoi93fBy24vtlDltcX2axwfWVJxtdX/lRpNeXUsoFI5nP0lovzKGJda8vS98YKMwXxieGU0AV/r4xUO+ONl25/abCTnOPtXJclYATQOs7HvcCfG75fivQqQjjKsvfC8hCgbPZ751N36/sdn4Y46BeRfF+3fIawdz7Jl+RX19mxlXk15eZcRX59WVOXLa4vrL/3jOAr3JpY9Xry2JvrgX/kbpg3B0+CbyZ/dgoYNQtb9qE7OcPACG5HVuEcf0EXAH2Zn+FZT9eNfsfZx9wyAZxjc1+3X0YN9Na53ZsUcWV/fMQYM4dx1n7/ZoNnAfSMXpFw4vJ9ZVXXLa6vvKKy1bXV65x2eL6whgG08D+W/6duhTl9SVL/4UQwkEUtzF0IYQQBSQJXQghHIQkdCGEcBCS0IUQwkFIQhdCCAchCV0IIRyEJHQhhHAQ/w9aIdoxn8o3xQAAAABJRU5ErkJggg==\n",
       "text/plain": [
        "<AxesSubplot:>"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
   ],
   "source": [
     "# The .figure() method will create a new figure, or activate an existing figure.\n",
    "plt.figure()\n",
    "# The .plot() is a versatile function, and will take an arbitrary number of arguments. For example, to plot x versus y.\n",
    "plt.plot(X, Y, label='actual')\n",
    "plt.plot(X, predict(Xf, W), label='predicted')\n",
    "# The .legend() method will place a legend on the axes.\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2021 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
